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ABSTRACT 


Pseudorandom  numbers  can  be  obtained  statically,  i.e.,from  stable 
of  random  numbers,  or  they  may  be  generated  continually  (dynamically). 
Static  cases  have  been  investigated;  e.g.,  tables  of  random  digits  often 
contain  results  of  tests.  The  dynamic  case,  however,  implies  that  tests 
for  randomness  be  made  continually.  Go/no-go  methods  are  attractive. 

Problems  associated  with  measuring  randomness  in  the  dynamic 
case  of  a  generator  continually  producing  binary  digits  are  investigated. 
The  mathematics  of  Bernoulli  trials  serves  as  the  model  against  which 
the  performance  of  the  generator  is  compared.  It  is  shown  that  there 
are  a  large  number  of  ways  in  which  a  measuring  system  might  be 
attempted.  Such  systems  are  based  on  explicit  functions  (called  measure 
functions)  of  p,  the  probability  that  a  *one”  is  generated. 

Since  the  digits  can  be  grouped  (a  group  consists  of  m  digits), a 
binomial  expansion  [p  -f  (l  -  p)]"  can  be  written  for  each  value  of  m. 
Any  term  or  combination  of  terms  of  any  expansion  can  serve  as  a 
measure  group,  the  basis  of  a  measure  function. 

A  typical  measuring  problem  involves  the  case  where  p  is  constant 
and  has  a  value  within  specified  tolerances.  A  go/no-go  measurement 
indicates  that  p  is  within  the  specified  tolerances.  Means  are  provided 
for  the  setting  of  measure  groiq) -count  t  .erances;  a  count  alone  then 
provides  the  indication  of  an  acceptable  or  nonacceptable  value  of  p. 
The  sample  size  is  selected  on  the  basis  of  the  desired  confidence  limit 
or  the  upper  bound  of  the  error  in  measuring  p. 

Since  there  are  many  possible  measure  functions,  some  means  is 
required  for  comparing  the  relative  effectiveness  of  different  functions. 
A  useful  method  of  comparison  is  available  if  measure  function 
■acceptance”  characteristics  are  plotted.  An  acceptance  characteristic 
is  a  plot  of  the  probability  that  the  measure  grotq)  digits  occur  within 
the  determined  group-count  tolerances.  The  acceptance  characteristic 
is  plotted  for  the  results  obtained  from  a  single  application  of  a  partic¬ 
ular  measure  function.  By  requiring  a  sequence  of  applications  of  the 
same  measure  function  and  by  introducing  an  acceptance  decision 
criterion,  the  acceptance  characteristic  more  nearly  iq)proaches  the 
ideal. 
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ABSTRACT  (Continued) 


Under  certain  conditions,  a  failure  of  the  generator  might  manifest 
itself  by  the  appearance  of  a  repeated  sequence  of  digits.  Some  of  the 
necessary  conditions  are  investigated  for  a  go/no-go  measuring  system 
to  breakdown,  i.e.,  to  give  erroneous  “accept"  or  “go"  readings  for  an 
input  consisting  of  specific  repeated  sequences .  It  is  found  that  immunity 
from  breakdown  is  dependent  upon  the  number  of  digits  in  the  group 
used  in  the  measure  function;  the  longer  the  group,  the  longer  is  the 
sequence  of  digits  which  c.an  cause  an  erroneous  indication  -  thus, 
more  immunity. 

The  various  aspects  of  the  problem  of  a  dynamic  measurement  of 
randomness  are  illustrated  in  an  arbitrary  example.  Results  are  obtained 
for  three  measure  functions. 

In  developing  the  theory,  a  method  for  measuring  p  was  obtained; 
it  provided  an  independent  method  useful  in  checking  the  go/no-go 
results . 


PROBLEM  STATUS 

This  is  an  interim  report  on  a  phase  of  the  problem;  work  is 
continuing  on  this  and  other  phases  of  the  problem. 
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SOME  ASPECTS  OF  GO/NO-GO  TESTING  OF  RANDOMNESS 
OF  CONTINUALLY  GENERATED  BINARY  DIGITS 


INTRODUCTION 

There  are  two  ways  in  which  random  or,  more  appropriately,  pseudorandom  numbers 
can  be  obtained.  The  numbers  can  be  taken  from  a  table,  i.e.,  statically,  or,  they  can  be 
created  continually  (dynamically)  and  used  as  required.  In  the  past,  a  number  of  (static) 
tabulations  satisfied  the  needs.  With  the  increased  use  of  high-speed  computers,  the  need 
for  generating  pseudorandom  digits  continually  (dynamically)  has  also  increased.  Routines 
are  available  which  can  be  used  by  specific  computers  to  create  pseudorandom  digits  as 
the  machine  requires  them.  There  are  other  means  for  creating  digits  randomly.*' 

Regardless  of  how  the  digits  are  generated,  the  problem  remains  to  determine  if  the 
digits  produced  are  random.  In  Ihe  static  case  of  a  tabulated  set  of  digits,  the  tabulation 
is  subjected  to  a  set  of  statistical  tests.**  The  fact  that  the  table  was  published  indicates 
that  the  conclusion  drawn  from  the  tests  is  that  the  entries  are  random  with  a  high  enough 
degree  of  confidence.  In  the  dynamic  case,  where  the  random  digits  are  produced  con¬ 
tinually,  repeated  testing  is  necessary  if  the  process  of  generating  digits  is  to  be  checked. 
In  the  case  of  a  computer  routine,  the  routine  is  selected  because  the  digits  produced  will 
exhibit  random  properties.  The  problem  of  testing  the  randomness  of  digits  produced 
continually  does  not  appear  to  have  been  explored.  It  is  this  problem  which  is  being 
investigated  in  this  report. 

We  will  concern  ourselves  more  specifically  with  the  problem  of  investigating  the 
measurement  of  randomness  of  continually  generated,  random,  binary  digits.  As  it  turns 
out,  there  are  a  large  number  of  ways  of  making  the  measurement,  and  each  method  can 
be  adapted  for  testing  any  set  of  random  numbers.  While  the  static  case  permits  a  more 
leisurely  iqiproach  to  the  measurement  of  randomness,  the  dynamic  case  presents  the 
unique  situation  of  continually  requiring  rapid  evaluations  of  the  numbers  being  generated. 
Because  of  its  attractiveness  for  the  indicated  purpose,  a  go/no-go  method  of  measurement 
is  being  investigated.  Since  go/no-go  methods  suggest  unattended  operation,  a  detailed 
examination  is  ^ipropriate  to  establish  the  likely  limitations  which  are  an  inherent  part 
of  the  method. 

It  will  be  shown  that  the  measuring  method  depends  upon  the  probability  p  that  the 
digit  generated  is  a  *one.”  There  are  a  variety  of  relationships,  explicit  functions  of  p, 
which  can  be  formed;  relations  called  measure  functions  are  derived.  The  manner  in 
which  meimure  functions  can  be  obtained  and  applied  is  indicated.  To  illustrate  the  {Appli¬ 
cation  of  the  me  (muring  technique,  three  measure  functions  are  treated  in  (ui  arbitrary 
problem  (arbitrary  bec(uise  (my  specific  problem  requires  a  prior  statement  of  the 
tolerances  in  the  value  of  p  which  will  be  permitted).  In  the  illustrative  example,  three 
measure  functions  are  compared,  and  it  is  concluded  that  effective  results  are  obtainable. 

As  a  subsidiary  result,  a  me(ms  is  obtained  for  the  measurement  of  p.  This  result 
provides  an  independent  method  which  can  be  used  in  checking  results  obtained  by  go/no- 
go  methods. 


*See  the  introductory  remarks  on  the  production  of  random  digits  and  the  tests  for  random¬ 
ness  in  the  Rand  Corporation's  "A  Million  Random  Digits,  with  100,000  Normal  Deviates,” 
The  Free  Press,  1955. 
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THEORY 


Introduction 


Let  us  examine  a  process  whose  output  is  a  sequence  of  binary  digits.  Suppose  we 
arrange  the  sequence,  as  it  is  created,  into  groups  of  m  digits  each.  If  we  list  the  m-digit 
groups  in  order,  as  they  are  generated,  we  can  form  for  n  such  groups  an  array  of 
digits  as  follows: 


where 


^12 

■  • 

bjj  bjj 

•  • 

^il  ^i2 

•  •  bi„ 

^nt  *^02 

•  •  bnm 

ran 

b.m  =  0or  1. 

II 

(1) 


(2) 


since  we  are  dealing  with  binary  numbers.  The  n  by  m  array  has  -n  rows  and  m  columns. 
Consider  any  arbitrary  column,  say  the  mth.  We  can  say  that 


2]  ‘’i-.  = 

i-i 

where  is  the  number  of  ones  in  the  column.  If  we  let  n  increase  without  limit,  then  we 
can  define 


im  i  bin,  =  p, 

^  00 

i»l 


i 

n  00 


(4) 


as  the  probability  of  a  one  occurring  anywhere  in  the  column.  Thus,  if  the  process  is 
statistically  stable,  or  n  is  large  enough. 


Since 

then 

or,  in  terms  of  p„  and  q„ , 


«m(l) 

— - Pm* 

«m(l)  +  "m(0)  =  n 
n„(0)  =  n-n„(l) 

‘Im  =  *  -  P«*  Pm  +  ‘lm  =  1 


(5) 

(6) 

(7) 

(8) 


where  q„  is  the  probability  of  a  zero  occurring  anywhere  in  the  column.  In  the  same  way, 
values  of  p„  and  q„  can  be  obtained  for  any  m .  In  fact,  a  value  of  p  can  be  obtained  by 
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extracting  a  sample  of  any  m  digits  from  the  array  given  in  (1).  In  such  a  case,  the  number 
of  ones  in  the  sample  is  defined  as  m(1)  ,  and 

p,  0  <  fji  <  n  .  (9) 

n 

We  can  see  that  (9)  gives  the  general  form  of  a  system  of  measurement  of  p.  However,  it 
involves  the  counting  of  digits,  or  summing  (derations,  and  the  operation  of  division.  It 
is  desirable  to  use,  if  possible,  only  the  summing  of  digits  in  a  system  of  measurement. 

Upon  examination  of  (1)  we  find  that  for  a  finite  value  of  n  and  for  a  given  array  we 
can  form  a  number  of  groupings,  each  of  which  contain  m  digits,  and  that  this  number  of' 
groupings  is  the  number  of  combinations  of  ran  things  taken  ^  At  a  time  (')1!').  Hence,  if 
we  take  a  large  enough  number  of  groupings,  we  obtain  a  sequence  of  numbers 


Mjfl).  Mjd).... 


(10) 


and  if  these  numbers  are  "statistically  consistent,”  we  might  conclude  that  there  is  a 
single  value  of  p  which  can  describe  the  digit  generating  process.  In  other  words,  we 
conclude  that  the  array  of  digits  was  produced  under  the  condition  that  p,  the  probability 
of  a  one  occurring  anywhere  in  the  array  is  constant.  We  are  drawn  to  the  conclusion 
that,  ideally,  the  digit  generating  process  is  exactly  that  of  Bernoulli  trials.  Consequently, 
we  look  to  the  binomial  distribution  to  guide  us  to  the  meaning  of  the  term  "statistically 
consistent”  as  used  with  (10).  The  binomial  distribution  is  given  by  ” 

B(M.k.p)  =  (j^)  p**  (k  :  0,1,2 . n)  (11) 

where  B(/i,k,p)  is  the  probability  that  in  a  sample  of  size  m,  exactly  k  ones  will  be  pre¬ 
sent;  i.e.,  k  =  m(1).  Using  (11)  we  can  calculate  the  variation  in  k  to  be  expected  in  the 
ideal  case.  A  basic  aspect  of  the  method  of  measuring  randomness  lies  in  the  comparison 
of  the  performance  of  an  actual  process  with  that  of  the  ideal  process  of  Bernoulli  trials. 


Measure  Groiqis 

In  (11)  the  function  involves  p  directly;  it  is  a  simple  function.  The  use  of  (11)  as  Just 
indicated  seems  to  imply  the  test  of  static  situations,  i.e.,  tests  using  a  single  array  as 
indicated  in  (10).  In  a  dynamic  case,  different  arrays  would  be  involved.  In  a  djmamic 
case,  either  a  single  test  or  a  number  of  tests- such  as  (10)  might  be  performed;  the 
implicit  assumption  is  made  that  p  remains  constant  (or,  equivalently,  that  p  remains 
wiftin  tolerable  limits)  for  each  array  tested. 

Returning  for  another  look  at  (1)  we  note  that  the  generated  sequence  of  digits  is 
arranged  in  groupings  of  m  digits  each,  fo  the  above  discussion  we  approached  the  array 
essentially  through  the  columns.  Now,  suppose  we  consider  the  rows.  With  m  digits  in 
each  row,  we  can  form  2”  arrangements  of  the  m  binary  digits.  From  entries  in  an  array 
we  can  form  digits,  in  essentially  a  new  number  system,  in  whatever  fashion  we  desire. 

If  we  care  to  do  so  for  any  particular  array,  we  find  that  we  can  form  (™)  combinations 
of  nm  things  taken  m  at  a  time.  If  we  wish  to  examine  a  sample  size  of  m  digits  in  the  trans¬ 
formed  number  system,  we  would  find  that  we  could  form  (IT))  samples.  In  this  case,  as 
in  ttie  previous  case,  a  static  analysis  can  be  made,  or  the  tealysis  can  be  made 
dynamically. 


'*'See  W.  Feller,  "An  Introduction  to  Probability  Theory  and  its  Applications,”  Vol.  1,  p.  106, 
New  York:WUey,  1950. 
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It  is  clear  that  no  restriction  has  been  placed  on  the  value  of  m.  This  discussion  indi¬ 
cates  that  we  may  exhaustively  test  arrays,  either  statically  or  dynamically.  We  will  not 
consider  any  further  here  the  extent  to  which  we  must  go  toward  exhaustiveness.  Instead, 
we  will  consider  what  can  be  expected  from  making  single  tests,  or  a  small  number  of 
tests. 

We  have  thus  far  indicated  a  simple  function  involving  p  itself.  We  also  indicated  that 
we  might  treat  groupings  of  the  digits  by  effectively  treating  the  digits  of  a  number  system 
with  a  higher  order  base,  or  radix,  2".  The  simple  function  consists  of  considering  single 
digits.  We  could  just  as  well  consider  2,  3,  4,  or  more  digits  together,  and  examine  their 
occurrences.  This  is  equivalent  to  taking  m  =  2,  3,  4,  etc.  By  the  same  process  which  led 
to  (8),  we  can  obtain  the  relations  for  the  probability  of  occurrence  of  specific  pairs, 
triples,  quadruples,  or  quintuples  of  digits  from  the  following  relations: 


q  +  P  =  1. 

(i"=  1) 

q*  +  2pq  +  p*  =  1. 

(m  =  2) 

q^  +  3pq^  +  3p^q  +  p^  =  1 . 

(m  =  3) 

q^  +  4pq^  +  6p*q*  +  4p^q  +  P*  =  1. 

(m  =  4) 

q®  +  Spq*  +  lOp^q®  +  10p®q^  +  5p*q  +  p® 

=  1 .  (m  =  5) 

and  so  on.  These  terms  are  obtained  from  the  binomial  expansion  of  (q  *.  p)". 

At  this  point,  let  us  consider  a  specific,  but  arbitrary,  choice  of  a  new  radix.  Suppose 
we  take  m  =  4.  Then  2"  =  2  and  we  have  a  radix  of  16,  or  what  is  often  referred  to  as 
the  hexadecimal  number  system.  While  we  will  use  this  radix  value  throughout  the 
remainder  of  this  report,  other  values  could  be  used  as  well.  For  this  case,  the  binary 
combinations  of  the  four  digits  are  shown  in  Fig.  1(b),  together  with  commonly  used 
hexadecimal  digital  equivalents  and  expressions  for  the  probabilities  of  occurrences  of 
each  of  the  hexadecimal  digits.  Several  possibilities  are  available  to  us.  We  can  use  any 
one  of  the  hexadecimal  digits  as  an  indirect  measure  of  p,  or  we  can  use  various  groupings 
of  these  digits.  Among  the  possible  groupings  of  digits,  we  can  treat  together  those  digits 
which  contain  various  numbers  of  ones.  In  this  way  we  can  form  five  groiq>s,  those  varying 
in  content  from  none  to  four  ones.  Forming  the  individual  probabilities  (Fig.  1(b)),  for  these 
groups  of  digits  and  taking  their  sum,  i.e.,  the  probability  that  any  of  the  groups  will 
occur,  we  write 


+  4pq^  +  6p*q*  +  4p*q  +  p^  =  1.  (13) 

We  can,  furthermore,  combine  terms  of  (13)  as  follows: 

(q^  +  4pq’)  +  (6p*q*)  +  (4p^q  +  p^  )  =  1 .  (14) 

We  see  that  there  are  a  variety  of  ways  of  forming  groups  of  hexadecimal  digits.  Any  one 
of  die  groupings  In  (13)  or  (14)  forms  a  valid  function  which  can  be  used  in  the  indirect 
measurement  of  p.  In  a  similar  way,  we  can  form  groups  of  digits  with  any  of  the  terms 
or  combinations  of  terms  given  in  (12).  The  actual  groupings  which  can  be  obtained  are 
shown  in 'Fig.  1.  It  is  possible  to  form  other  groupings  by  taking  higher  values  of  m. 

Since  there  are  a  number  of  choices  to  be  made  with  regard  to  possible  groupings,  we 
will  have  to  consider  ways  of  comparing  the  results  obtained  by  using  a  number  of  the  func¬ 
tions;  the  functions  we' use  will  be  called  measure  groups.  We  will  consider  three  measure 
groups 'later  as  part  of  an  illustrative  example.  Now  we  wish  to  see  how  we  can  adapt  any 
of  these  functions  to  a  scheme  of  measurement. 
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Fig.  (1)  -  Schematic  or  tree  of  (a)  one-,  two-,  and  three-bit 
processes  (  •<>=  1,2,3),  (b)  A  four-bit  process  (>n  =  4),  and 
(c)  a  five-bit  process  (in=  5) 


We  have  already  indicated  how,  with  (11)  and  the  function  ,  comparisons  can 

be  made  with  the  ideal  case  or  model  -  only  the  count  of  ones  is  required.  The  use  of 
(11)  applies  as  well  to  the  case  where  compound  functions,  as  in  (13)  or  (14),  are  involved. 
Let  us  consider  the  first  term  of  (13)  as  a  measure  group;  it  is  a  compound  function  of 
p  because  it  is  composed  of  four  digits,  which  are  all  zeros.  Let 


P,  =  q"  (15) 

be  the  probability  that  the  binary  group  0000,  or  the  hexadecimal  digit  0,  has  occurred. 
Then 


dg  =  1  -  =  1  -  pg- 


(16) 
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OCCURRENCE  OF 
DIGITAL  GROUPS 
OR  SYMBOLS 

f 

P4 

• 

pS  q 

d 

pSq 

c 

p2q2 

b 

pSq 

0 

p2q2 

9 

P*q* 

8 

pqS 

7 

pSq 

6 

p2q2 

5 

P*q* 

4 

pqS 

3 

pZq2 

2 

pqS 

1 

pqS 

0 

q^ 

Fig.  1  (Continued)  -  Schematic  or  tree  of  (a)  one-,  two-,  and 
three-bit  processes  (m  =  1,Z,3),  (b)  a  four-bit  process  (ni=  4), 
and  (c)  a  five-bit  process  (m=  5) 
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From  (13)  and  (16) 


‘Ig  =  (4pq^  +  6p^q*  +  4p^q  +  p^)  (17) 

or,  the  probability  that  the  measure  group  has  not  occurred  is  just  the  probability  that  any 
one  of  the  other  groups  has  occurred.  This  leaves  us  with  a  binary  situation,  which  can  be 
treated  using  the  binomial  law.  Thus,  a  measure  group  can  be  transformed  into  a  new 
variable  and  its  complement  by  (15)  and  (17).  Note  that  in  a  dynamic  situation  both  p,  and 
q^  will  be  constant  as  long  as  p  is  constant  with  respect  to  time. 

We  can  rewrite  (11)  for  the  new  variable  and  for  a  sample  size  of  n  as 

B(n,k.Pj)  =  p^  qj""'‘  (18) 

for  which  can  be  obtained  the  probability  that  there  will  be  exactly  k  occurrences  of  the 
particular  group  out  of  a  sample  of  n  trials.  We  can  think  of  k  as  being  the  reading  of  a 
counter  which  gives  the  equivalent  value  of  n(K),  the  number  of  expected  occurrences  of 
the  special  measure  group  chosen.  Thus,  k  becomes  a  measure  of  p^.  However,  we  can 
only  conclude  from  a  given  reading  which  falls  within  the  range  of  expected  values  of  k 
that  the  process  is  following  the  model  or  ideal  case.  We  can  visualize  a  particular  test 
situation  in  which  both  n  and  k  are  fixed,  and  we  do  not  know  the  value  of  Pg.  We  will  find 
that  there  can  be  a  range  of  values  of  Pg  which  could  probably  have  given  the  specific 
counter  reading.  We  seek  a  range  of  values  of  k  which  will  permit  us  to  conclude  that  Pg 
is  within  allowable  tolerances.  We  are  concerned  in  (18)  with  three  variables;  n,  k,  and  p,. 


Upper  Bound  of  Error  in  p 


Since  we  made  use  of  a  limiting  process  in  defining  p  in  (4),  it  is  appropriate  to  turn 
to  the  law  of  large  numbers.  We  seek  a  relation  between  n,  k,  and  p,.  The  law  of  large 
numbers  may  be  expressed'"  as  follows: 


(19) 

(20) 


that  is,  certainty  as  n  increases.  In  these  statements  c  is  a  preassigned  small  number, 
selected  suitably  to  the  conditions  of  the  problem,  which  represents  the  measuring  error 
or  difference  to  be  expected  between  k/n  and  p,.  The  right >hand  side  of  (19)  is  the  dif¬ 
ference  between  two  values  obtained  from  tables!  of  the  normal  distribution  function  (or 
the  normal  cumulative  distribution  function).  The  terms  of  (19)  are  of  the  form 


♦(*)  =  j*  0(t)dt 

(21) 

where 

(22) 

is  the  normal  density  function. 

♦Feller,  op.  cit.,  p.  137. 
top  cit.  pp.  136  and  137. 
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The  difference  on  the  right-hand  side  of  (19)  is  called  the  confidence  limit  and,  as  given, 
shows  that  equal  areas  of  the  distribution  function  are  excluded  at  either  end  (i.e.,  extreme 
positive  and  negative  values  are  excluded).  For  the  case  where  a  95  percent  confidence 
limit  is  desired,  95  percent  of  the  area  under  the  distribution  function  is  taken  into  con¬ 
sideration.  Furthermore,  half  the  difference,  or  2.5  percent  of  the  area,  is  excluded  at 
either  end.  Thus,  when  tables  of  4(x)  are  available. 


$  ji  i  +  0.025  =  0.975 

hence. 


(J3) 


,  i  (34) 

Therefore,  we  can  tabulate  the  following  values  of  n  and  e  (both  numeric)  for  assumed 
values  of  p,  with  95  percent  confidence: 


n  e 


100 

0.1 

1,000 

0.03 

10,000 

0.01 

100,000 

0.003 

The  effect  of  choosing  different  confidence  limits  is  shown  in  Fig.  2  for  values  of 
P,  between  0.3  and  0.7.  The  use  of  Fig.  2  permits  one  to  make  a  graphical  selection  of  n, 
once  the  error  in  measuring  p.  has  been  selected.  Using  the  tqppermost  line  in  Fig.  2  for 
selecting  n  provides  a  reasonable  upper  bound  for  the  error. 

Use  of  Fig.  2  permits  the  selection,  in  practice,  of  a  suitable  measuring  error.  The 
desired  error,  however,  is  dial  Inp,  rather  than  in  p  .  The  relation  between  the  relative 
error  in  p.  and  that  of  the  relative  error  in  p  is  derived  in  Appendix  B.  It  is  shown  that 
Fig.  2  still  serves  a  useful  purpose  in -determining  the  magnitude  of  the  likely  measuring 
error  to  be  encountered  and  still  permits  the  selection  of  value  of  n. 


Measure  Function 

Next,  as  a  means  for  approximating  the  eiqiected  limits  of  k ,  use  can  be  made  of  the 
De  Moivre-Laplace  limit  theorem*  which  states  the  equivalence 


ni'nlrn'l-  ^ 

B(n,k.p  )  - 1-sas.exp-  - S —  , 

/2Trnp  q 


(25) 


The  equivalence  given  in  (25)  is  subject  to  the  condition  diat 

(k-np,)^ 


-0. 


or 


n**  -0. 


(26) 


In  (25)  and  (26),  np,  is  the  average  number  of  occurrences  of  the  particular  grouping, 
(k-np,)  is  a  deviation  from  the  average  value,  and  np^q,  is  the  variance  (er^). 


*W.  Feller,  op.  cit.,  pp.  133-137. 
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NUMBER  Of  TRIALS.  OR  SAMPLE  SIZE,  n  (NUMERIC) 

Fig.  2  -  The  measuring  error  as  a  function  of  the  number  of  trials. 
For  a  choice  of  confidence  linoit.  selecting  €  will  give  n. 


By  making  the  substitutions 


(27) 

and 

x^  = (k-np,)h 

(28) 

we  can  rewrite  (25)  in  the  form  of 

B(n,k,pj)  =  h(f>(x^) 

(29) 

where 

(30) 

is  the  normal  density  function.  It  can  be  shown'*'  that  (29)  can  be  expressed  with  little 
error  by 

nth/ 2 

h<#i(Xfc)%  dt  =  $  (x^  ^,/j)  -  ♦  (Xfc.j/j).  (31) 

«l,-h/2 

The  numerical  evaluation  of  (18)  is  possible  with  the  aid  of  (31)  and  tabulations  of  the 
normal  distribution  function.  We  have  inaicated  that  we  will  be  concerned  about  a  range  of 
values  of  k.  Therefore,  we  are  concerned  about  the  function 


2 


*yf.  Feller,  op.  cit.,  p.  137. 
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which  gives  the  probability  that  the  reading  will  be  in  the  interval  1  k  1  k, ,  this  func¬ 
tion  is  called  a  measure  function.  It  can  be  showni*  that 

M<ki.k  J.n,p,)  =  $  ^  .  (33) 

With  the  aid  of  tables  of  the  normal  probability  function  (if  necessary,  transformations 
given  in  Appendix  A  can  be  used),  a  good  approximation  of  (32)  is  obtained.  It  is  interest¬ 
ing  to  note  that  condition  (19)  is  a  special  case  of  (33). 


Validity  Condition 

In  choosing  specific  values  of  x,  care  must  be  taken  to  keep  within  condition  (26)  on 
which  the  normal  law  approximation  was  based.  Condition  (26)  may  be  restated  as 

Xl,*hp‘,*q,2  0.  (34) 

Condition  (34)  establishes  the  validity  of  the  application  of  the  De  Moivre-LaPlace  limit 
theorem  (25)  and  (33).  Fellert  demonstrates  that  (34)  is  satisfied,  in  many  applications, 
for  values  of  x,,  of  3  or  4.  With  x,,  between  3  and  4  and  n  between  10  and  100,  the  limit 
of  acceptability  can  be  written  as 


x^-*hp,2q^J  <  1 .  (35) 

It  follows  that  a  maximum  usable  value  of  xi^  is 

X|,3  S  h*n*  or  X|,  S  h(n)*^®.  (36) 

Note  ttiat  for  larger  values  of  ,  the  left-hand  side  of  (35)  becomes  much  less  than  1  with 

*k  =  *• 


Count  (k)  Tolerances 

Each  term  on  the  right-hand  side  of  (33)  is  a  normal  distribution  function,  and  (36) 
gives  half  the  range  for  which  reasonable  approximations  are  valid.  The  fact  that  ttese 
terms  are  normal  distribution  functions  can  be  used  to  some  advantage  when  it  is  found 
necessary  to  go  beyond  the  validity  range  given  by  (36).  This  fact  will  become  evident 
as  the  discussion  proceeds. 

As  a  first  approximation,  the  special  value  x,,  =  0  can  be  used  in  setting  tolerances, 
i.e.,  in  finding  values  for  k.  and  k..  When  x,^  =  0,  it  follows  from  (28),  and  because  h  /  0, 
that 


k  =  k,  =  npj  . 

(37) 

Specifically,  the  assigned  limits  will  be  the  lower  limit 

and  the  upper  limit 

kl  =nP,, 

(38a) 

'■'Feller,  op.  cit.,  p.  137. 
top.  cit.  pp.  136  and  137. 

k,  =np^ 

(38b) 

12 


NAVAL  RESEARCH  LABORATORY 


where  p,  and  p,  are  either  given  directly  or  must  be  calculated  from  given  values  of  p. 

In  the  latt'er  casef  the  transformation  between  p  and  p,  can  be  performed  graphically  with 
the  aid  of  Fig.  3,  or  an  expanded  version  of  p^  vs  p  for  the  specific  measure  group, 
similar  to  Fig.  4. 

The  fact  that  both  terms  of  (33)  are  normal  distribution  functions  permits,  with  the 
aid  of  probability  graph  paper,  the  use  of  gr^hical  techniques  in  adjusting  the  tolerances. 

It  is  possible,  also,  to  show  graphically  the  expected  range  of  variation  in  k  about  a  selected 
tolerance.  By  solving  (28)  for  k  and  using  (27),  we  obtain 

k  =  npj  ±  X|j  a  =  k^  ±  X,,  cr*  (39) 


p  (NUMERIC)  p  (NUMERIC) 

(p«  PROBABILITY  THAT  A  I  OCCURS) 

Fig.  3  -  Graphical  representation  of  transformations  between  p  and  P,  for 
the  measure  groups  (values  of  m,  or  number  symbols)  indicated 


NAVAL  RESEARCH  LABORATORY 


13 


06 


05 


04 

S 

oe 

ui 

I  03 

z 

02 


01 


040  042  044  046  048  050  052  054  056  058 

p  (NUMERIC) 

L  -  -  L _ I _ I _ I _ ^ _ \ _ I _ I _ L- 

'OlO  -008  -  006  -004  -002  0  0  02  0  04  006  008 
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Fig.  4  -  Expanded  graphical  representation  of  tranformations 
between  p  and  p  for  sonoe  number  symbols  for  m  =  1  and  m  = 

4.  Thetwodotted  lines  show  one  possible  alternate  combina¬ 
tion  of  the  group  probabilities  for  m  '  4. 

In  (39),  since  p,  and  n  (hence,  (r  as  well)  are  constant,  the  range  of  values  of  k  can  be 
obtained.  In  Pig.  5  the  variation  in  k  is  shown  for  seven  values  of  X|,  (0,  ±1,  ±2, 
and  ±3)  for  each  of  three  values  of  n  (10,100  and  1000).  For  each  set  of  curves  the 
maximum  usable  values  of  x,,  are  shown.  In  only  the  case  for  n  =  10  does  the  extreme 
curve  (x),  s  3)  exceed  the  limiting  condition  of  (35). 


060 

_J 

0.10 


Tolerance  Lines 

In  evaluating  (33)  it  is  desirable*  to  use  the  limits  ±  h/2.  It  follows  from  (28)  that 
*k,  - 1  /2  =  («P*,  -  l/2)h  =  (k ,  -  1/2  )h  (40a) 

and 

“k,  + 1/2  =  ("Pgi  +  l/2)h  =  (kj  +  l/2)h.  (40b) 

The  values  given  by  (40)  will  be  used  in  (33). 

Having  a  knowledge  of  the  range  of  valid  values  of  x,,  we  can  obtain,  using  (28),  the 
expected  range  of  values  of  Pg  which  could  be  used  in  determining  their  contribution  to 
accept  readings.  Note  that  the  only  condition  imposed  on  Pg  thus  far  has  been  that  of 
constancy.  By  solving  (28)  for  Pg  we  obtain,  dropping  the  subscripts  of  both  k  and  x. 


Pg 


2k 

2(x2  +  n) 


x*  +  2k 

*  k* 

2(x2  +n) 

h(x*  +  n)  ' 

(41) 


the  range  of  values  of  Pg  to  be  expected  for  given  n,  k,  and  x.  It  is  expected  that  (40)  will 
be  used  to  provide  values  for  x  and  k  for  (41).  The  sign  of  the  radical  in  (41)  is  chosen  on 
the  basis  of  the  expected  value  of  Pg;  there  is  symmetry,  of  course,  about  x  =  0. 


*W.  Feller,  op.  cit.,  p.  137. 
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Fig.  5  -  Group  count  (k)  as  a  function  of  P|,  the  probability 
of  occurrence  of  the  particular  digital  group  or  number 
symbol,  for  n  «  10,100,1000,  and  for  *k  =  0,  11,  ±2,  axid  13 
for  each  value  of  n. 

A  knowledge  of  the  range  of  valid  values  of  p,  is  useful  for  several  reasons.  It  pro¬ 
vides,  first  of  all,  a  test  of  the  range  of  significant  values  over  which  the  probability 
that  counts  within  the  tolerance  limits  will  occur.  In  addition,  the  valid  regions  of  p,  for 
both  terms  of  the  right-hand  side  of  (33)  can  be  obtained  independently.  Furthermore  (41) 
is  the  basis  for  a  method  of  comparing  different  measure  functions,  as  will  be  shown  in 
the  sequence. 

By  using  (41)  to  calculate  at  least  two  points,  a  straight  line  can  be  drawn  on  arith¬ 
metic  probability  graph  paper  for  each  of  the  terms  on  the  right-hand  side  of  (33);  these 
lines  we  call  tolerance  lines.  The  tolerance  lines  will  be  found  to  be  useful  in  construc¬ 
tion  of  the  acceptance  characteristic  of  measure  functions,  as  will  be  shown. 


An  Independent  Measure  System 

With  the  aid  of  Fig.  5  we  have  another  measuring  system.  This  system  yields  values 
of  p  and  consists  of  obtaining  a  set  of  readings.  Each  reading  is  obtained  from  an  n  byn 
array .  To  obtain  each  reading,  a  count  is  made  of  the  frequency  of  occurrence  of  the 
specific  arrangement  of  digits  indicated  by  the  selected  measure  group  e.g.,  (IS).  By 
taking  a  sufficiently  large  number  of  readings,  calculation  of  the  average  value  of  count  will 
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permit  the  use  of  Fig.  5  in  obtaining  the  most  likely  value  of  p,  (the  x,,  =  0  line  is  used). 
Associated  with  the  determination  of  the  most  likely  value  of  p^,  there  is  an  indication  of 
the  range  of  variation  in  count  to  be  expected,  and  the  set  of  readings  can  be  tested 
against  this  variation  for  consistency. 


THE  GOAJO-GO  measuring  SYSTEM 

While  the  aforementioned  procedure  does  provide  a  measuring  system,  it  is  not  a 
go/no-go  method;  an  average  value  must  still  be  calculated.  The  advantage  of  a  go/no-go 
method  of  measurement,  in  which  only  counting  in  involved,  lies  in  the  simplicity  of  inter¬ 
pretation.  Any  particular  reading,  when  obtained,  carries  with  it  an  accept  or  reject 
classification.  This  type  of  classification  tends  to  permit  the  conclusion  that  when  a  “go” 
or  accept  classification  is  obtained,  the  process  is  following  the  model,  and  the  value  of 
p,  is  within  the  tolerance  permitted.  To  assure  ourselves  of  this  conclusion,  we  must 
examine  the  expected  results  more  fully.  In  a  go/no-go  measuring  system,  allowable 
tolerances  on  p  will  be  set,  n  will  be  determined,  and  the  tolerances  on  k,  k^,  and  kj  will 
be  obtained,  at  least  initially. 


The  Effectiveness  of  Measure  Functions 

Equation  (41)  is  of  value  in  the  comparison  of  specific  measure  functions.  For  example, 
one  method  of  comparison  might  consist  of  calculating  the  value  of  pg  for  some  “standard” 
value"'  of  X.  By  transforming  the  value  of  p,  into  p,  a  range  of  values  of  p  which  provide 
accept  readings  is  obtained.  The  problem  of  comparing  different  measure  functions  leads 
us  to  consider  the  ideal  measuring  problem.  Fig.  6.  The  measure  area  of  Fig.  6  is  a  plot 
of  mOc}  I  kj  ,  n,Pg )  vs  p.  It  can  be  shown  that  both  scales  have  minimum  values  of  0  and 
maximum  values  of  1.  Ideally,  an  accept  region  a  which  is  sharply  divided  from  the 
reject  region  is  desired.  We  will  find  in  practice  a  measuring  situation  more  like  that 
shown  in  Fig.  7,  where  the  lines  of  demarcation  of  the  accept  region  are  not  linear.  Note 
that  the  extent  of  die  accept  region,  the  p-tolerance  region,  is  ideally  2Ap  or  ^Px  -t-  ^P2  , 
depending  iqion  whether  or  not  the  tolerances  are  assigned  symmetrically.  This  ideal 
situation  suggests  that,  in  terms  of  p-tolerances. 


[i  “O'l.ka.n.p,)]  ^  ^ooand  [^ll(kx 


.k 


,.n.p,)] 


Po  +  Pj 


or,  equivalently,  in  terms  of  p^ -tolerances. 


(42a) 


••(kj.kj.n.Pglj  and  |^M(k,,kj,n,Pj)j  ~  oo  (42b) 

where  the  derivative  is  evaluated  at  the  values  given  outside  of  the  brackets.  Condition 
(42)  can  be  expressed  more  practically  as  follows:  Among  alternative  measure  functions 
the  most  effective  is  that  whose  slope  is  greatest  at  each  tolerance  limit. 


In  Fig.  7  the  departure  from  the  ideal  is  indicated  by  the  two  cross-hatched  regions 
a,  and  r,.  The  region  a,  is  an  addition  to  the  accept  region  and  includes  values  of  p  out¬ 
side  of  the  interval  of  acceptance  (p^,  -  Ap.  ,  p^  +  Apj).  The  region  r,  is  an  addition  to 
the  reject  region  even  though  the  values  of  p  are  within  the  accept  interval.  The  extent 
of  these  two  regions  in  terms  of  values  of  p,  can  be  obtained  using  (41)  for  both  values  of  k. 


"■The  choice  of  a  standardized  value  is  quite  arbitrary.  The  value  of  x  which  is  chosen 
should  be  large  enough  to  include  a  reasonable  range  of  values  of  p  and  still  be  con¬ 
sistent  with  the  validity  condition  given  by  (35). 
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SCALe  OF  p  (NUMERIC) 


Fig.  6  -  The  ideal  measure  area 
VI  p)  with  the  accept  region 
linearly  separated  from  the  reject 


Fig.  7  -  A  typical  measure  areafiKp^)  vs  p) 
with  the  accept  nonlinearly  separated  from 
the  reject  region.  The  region  a,  is  an  addi¬ 
tion  to  the  accept  region  and  includes  values 
of  p  outside  the  interval  of  acceptance 
(p^-Apj.  p^+Apj).  The  region  is  an 
addition  to  the  reject  region  even  though  the 
values  of  p  are  within  the  accept  interval. 


SCALE  OF  p  (NUMERIC) 


In  terms  of  these  areas,  condition  (42)  can  be  restated  as  follows:  (a)  if  the  value  of 
pto  be  determined  is  in  the  accept  intervsd  (p^  -  Ap^ ,  t  Ap,),  the  best  measuring 
function  is  that  which  gives  the  smallest  area  r,;  (b)  il  the  value  of  p  is  outside  the  accept 
interval,  the  best  measuring  function  is  Uiat  which  gives  the  smallest  area  a,. 

There  may  be  cases  where  either  one  of  the  above  two  conditions  can  be  satisfied.  In 
other  cases  a  compromise  may  have  to  be  made.  In  making  the  compromise,  a  measure¬ 
ment  of  the  areas  is  a  quantitative  indication  of  the  effect  to  be  expected  on  the  overall 
measurement.  In  any  event,  these  areas  are  of  iihportance  in  judging  the  e^qpected  effec¬ 
tiveness  in  the  determination  to  be  made  of  values  of  p. 

If  we  denote  the  ideal  accept  region  (Fig.  6)  by  A,  we  see  that 

A  =  2Ap  =  Apj  +  Apj  (43) 

since  the  ordinate  is  unity.  If  we  take  R  to  be  the  rejection  region,  then,  since  we  have 
unit  measure  area. 


(44) 


From  Fig.  7  we  have  an  effective  accept  region  A,  given  by 

A,  =  A  -  r,  +  a, .  (45) 

There  is,  correspondingly,  an  effective  reject  region  R,  given  by 

R,  =H-a,  +  r,.  (46) 

From  (45)  and  (46)  we  can  see  diat  if  the  areas  a  and  r,  are  made  equal  we  have 
A,  =  A  and  R,  =  B .  This  indicates  the  best  compromise  that  can  be  expected.  Note 
that  ilboth  tolerances  are  assigned  in  accordance  with  (37),  then  a  “best  conq)romise” 
results. 
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We  can  compare  various  measure  functions  on  the  basis  of  the  capability  to  give  accept 
indications  when  the  value  of  p  to  be  determined  is  within  the  tolerance  interval.  We  can 
also  make  a  comparison  on  the  basis  of  giving  reject  indications  vrhen  the  value  of  p  being 
determined  is  outside  of  the  tolerance  interval.  We  can  take  E,  as  a  measure  of  the 
acceptance  effectiveness,  where 


A-r  r,  r, 

*'•"  A  Apj  +  Apj 


(47a) 


and  E,.  for  the  rejection  effectiveness,  where 


E,  = 


R  -  a,  a 


=  1--^  =  !- 


■ 

1  -  (Api  +  Apj)* 


(47b) 


The  values  E,  and  E,  are  overall  indications  covering  all  values  of  p  either  within  the 
tolerance  interval  or  outside  of  it. 


Acceptance  Characteristics 

In  discussing  the  effectiveness  of  measure  functions,  the  line  of  demarcation  between 
the  accept  and  reject  regions  was  considered  in  a  general  way.  It  is  possible  to  obtain 
data  for  plotting  ftls  line  and  thereby  obtain  the  accept  reject  region  boundaries.  In  the 
case  where  tolerances  on  p  are  given,  die  characteristic  of  interest  is  a  plot  of  ll(kj,k2, 
n,p,)  vs  p.  A  plot  of  this  characteristic  for  each  of  several  possible  measure  functions 
will  provide  a  graphical  means  of  conyiarison. 

There  are  several  ways  of  obtaining  data  for  plotting  the  accept  reject  boundary  lines, 
i.e.,  the  acceptance  characteristic.  Once  n  is  obtained,  e.g.,  from  Fig.  2,  k|  and  k,  are 
obtained  from  (38).  Then  either  (18)  or  (33)  can  be  used.  Calculations  using  (18)  or  (32) 
can  be  made  either  directly,  or  tables*  can  be  used.  If  tables  are  used,  there  is  a 
restriction  on  values  of  n;  the  tabulatimi*  used  includes  values  of  n  iq)  to  1,000. 

The  use  of  (33)  with  due  regard  for  its  range  of  application  provides  a  practical  means 
of  getting  the  dsda  for  acceptance  characteristics.  In  fact,  the  two  terms  on  the  right-hand 
side  of  (33)  can  be  treated  independently.  It  is  necessary  to  treat  them  separately  if 
gross  errors  resulting  from  failure  to  satisfy  condition  (38)  are  to  be  avoided,  lliat  gross 
errors  can  be  avoided  becomes  evident  once  it  is  considered  that  (32)  and  (33)  both  attain 
the  values  0  and  1  for  finite  values  of  x .  It  is  possible  to  use  arithmetic  probability  graph 
paper  to  get  a  reasonably  approximate  accei;>tance  characteristic.  In  this  case,  wiA 
Pg  plotted  as  the  abscissa,  a  straight  line  can  be  drawn  connecting  the  selected  points. 

The  actual  characteristic  as  a  function  of  p  can  then  be  obtained  graphically. 

The  advantage  of  a  graphical  method  lies  in  the  ease  with  which  adjustments  can  be 
made  in  die  limits  initially  chosen.  The  comparative  effect  on  the  measuring  effectiveness 
can  be  seen  by  plotting  the  initial  and  subsequent  limit  lines. 

The  acceptance  characteristic  yields  at  least  a  qualitative  Judgment  of  the  measuring 
effectiveness  of  specific  measuring  functions,  n  the  p-tolerance  lines  are  drawn  in  and  the 
appropriate  areas  are  measured,  then  there  is  a  quantitative  Judgement.  In  the  measuring 
case  where  p  is  obtained  from  Fig.  5,  the  probable  range  of  variation  to  be  eiqiected  in  k 
is  obtained  directly  from  Fig.  5.  In  the  go/no-go  case,  a  point  on  the  acceptance 
characteristic  gives  the  probability  of  acceptance  for  the  particualr  value  of  p.  The  area 


Tables  of  the  Cumulative  Binomial  Probability  Function,”  by  the  Staff  of  the  Computational 
Laboratory,  Harvard  U.  Press,  1955. 
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under  the  acceptance  characteristic  represents  the  probability  of  acceptance  for  all  values 
of  p.  The  effective  accept  region  given  by  (45)  is  the  probability  that  if  the  value  of  p  being 
measured  is  within  the  tolerance  limits  for  p,  an  accept  indication  will  be  obtained. 

From  Fig.  7  we  see  that  the  probability  of  acceptance  decreases  as  we  increase  or 
decrease  p  with  respect  to  the  central  value  p^.  We  must  take  this  variation  into  account 
in  interpreting  the  readings  of  a  go/no-go  measuring  system. 


Effect  of  Repeated  Measurements  with  a  Decision  Criterion 

In  a  go/no-go  measuring  system,  the*value  of  p  is  not  obtained.  Instead,  the  measure¬ 
ment  indicates  that  a  particular  value  of  p  is  essentially  within  the  p-tolerance  interval. 

In  such  a  system,  the  acceptance  characteristic  gives  the  probability  of  acceptance  for  any 
particular  value  of  p .  The  value  of  the  probability  of  acceptance  remains  constant  as  long 
as  p  remains  constant.  Thus,  a  pseudorandom  digit  generator  which  is  stable  will  be 
ejqiected  to  yield  a  value  of  p  which  remains  substantially  constant.  If  for  some  reason 
a  change  in  p  occurs,  then  there  is  a  corresponding  change  in  the  probability  of  acceptance. 
If  the  value  of  p  was  initially  near  the  central  value,  then  the  probability  of  acceptance 
would  decrease  or  the  probability  of  rejection  would  increase.  Now,  we  can  choose  to  use 
either  the  probability  of  acceptance  or  of  rejection  as  the  constant  probability  in  a  new 
sequence  of  repeated  trials.  Let  there  be  n,  trials  with  the  constant  probability  p,  (i.e., 
the  probability  of  acceptance)  in  each  trial.  We  use  a  small  value  for  n,  and  assign  a 
value  to  kj,  the  number  of  acceptable  occurrences  out  of  trials.  The  expectation  that 
out  of  n,  trials  there  will  be  at  least  k^  occurrences  when  the  constant  probability  of 
acceptable  occurrence  in  a  single  trial  is  p„  is  given  by 


«• 

B(n,.kj.p,)  =  P.*  (1  -P.)"'~“  (48) 

The  probability  B(n,  ,l^,p,)  is  tabulated;  it  is  the  cumulative  binomial  probability  function 
referred  to  earlier.  Tne  selection  of  n  and  k^  is  quite  arbitrary.  For  purposes  of 
illustration  let  us  take  some  small  numoer  su^  as  n,  =  10.  With  the  aid  of  tables,  k^ 
might  be  so  chosen  as  to  make 

B(n,,k,,,p,)  i  0.50,  when  p,  =  0.50  .  (49) 

This  choice  of  p,  follows  from  the  way  in  which  kj  and  kj  were  selected  (by  (38)).  The 
result  from  the  tables  gives  k^  =  5  and  B(n,,kj,p,)  =  0.62. 

By  introducing  (48),  we  are  provided  with  a  means  of  interpreting  a  series  of  readings; 
in  short,  we  have  an  acceptance  decision  criterion.  With  its  aid  we  can  predict  the 
increase  in  rejection  rate  as  the  value  of  p  departs  from  the  central  value.  To  illustrate 
a  possible  use  of  this  type  of  decision  criterion,  the  function  (48)  has  been  plotted,  with 
the  aid  of  tables  for  four  values  of  k^  (Fig.  8).  Since  the  values  of  k^  and  k,  were  set 
essentially  to  give  a  probability  of  acceptance  of  50  percent,  if  we  use  (49)  as  the 
decision  criterion  then  whenever  p  sq)proaches  one  of  its  tolerances  it  can  be  expected 
that  there  will  be  only  about  5  acceptance  readings  in  10  trials.  From  Fig.  8  with  p.  = 

0.5,  we  could  expect  a  variation  in  the  number  of  acceptances  of  between  at  least  4  and  7, 
and  the  process  under  test  would  still  be  following  the  model. 


Repetitive  Patterns 

The  go/no-go  measurement  of  randomness  in  a  system  which  has  no  memory  involves 
the  risk  of  accepting  sequences  which  are  repetitive.  The  very  fact  that  a  given  random¬ 
ness  generator  begins  to  produce  sequences  of  digits  which  are  repetitive  is  an  indication 
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Fig.  8  -  Acceptance  criterion  [B(n,,lc4, 
p*)]  vs  the  probability  of  acceptance  for 
four  value  ■  of 


of  failure  of  the  generator  to  perforin  properly.  Thus,  it  is  the  capability  of  tiie  measuring 
system  to  detect  certain  t3rpes  of  generator  failure  that  we  are  investigating  at  the  moment. 
In  specific  cases,  a  generator  may  produce  only  certain  sequences,  so  with  these  known, 
the  measuring  system  should  be  designed  to  cope  with  tiiose  sequences,  i.e.,  to  give  reject 
indications  when  they  do  occur.  Here  we  point  out  a  few  of  the  general  aspects  of  the 
problem. 

The  problem  of  determining  the  effect  of  repeated  sequences  is  treated  in  detail  in 
Appendix  C.  It  is  shown  that  the  investigation  is  tedious  and  no  general  method  of  making 
the  test  is  evident. 

It  is  possible  to  make  the  following  qualitative  generalisations  from  the  investigation 
made  in  ^pendix  C. 

(a)  The  vulnerability  of  measure  functions  to  erroneous  accept  indications  depends 

upon  the  choices  made  for  ■  (i.e.,  the  specific  measure  groups  resulting  when  p  is 
designated),  as  well  as  iqpon  the  values  of  kj  and  k,.  ‘ 

(b)  While  some  reduction  in  vulnerability  to  accept  indications  is  possible  through  a 
narrowing  of  the  acceptance  limits,  the  change  in  the  acceptance  characteristic  should  be 
determined. 

(c)  The  use  of  a  decision  process  (e.g.,  multiple  application  of  a  measure  function) 
should  reduce  the  probability  of  erroneous  acceptance  indication  and,  therefore,  the 
vulnerability. 

(d)  It  should  be  possible  to  reduce  vulnerability  by  the  use  of  two  or  more  measure 
functions  which  individually  have  different  regions  of  vulnerability. 

That  the  incidence  of  repetitive  sequences  is  a  departure  from  the  ideal  can  be  seen 
if  die  probability  of  occurrence  of  specific  sequences  is  considered.  A  sequence  of  n, 
repeated  digits  can  be  expected  to  occur  with  a  probability  p,  in  such  a  way  that  in  the 
ideal  case  of  p  s  0.5 


P,  = 


(50) 


20 


NAVAL  RESEARCH  LABORATORY 


In  other  words,  each  of  the  possible  arrangements  is  equally  likely.  We  can  obtain  from 
tables  the  probability  that  for  a  given  m^a  specific  grouping  would  occur  continually  in  an 
ideal  process.  These  values  have  been  tabulated  (Table  1).  It  can  be  seen  that  for  m,  =  i, 
the  probability  that  there  will  be  a  sequence  as  long  as  8  digits  is  quite  small;  we  should 
never  expect  to  find  as  many  as  18  digits  repeated.  In  the  same  way  we  find  that  we 
should  encounter  almost  no  occurrences  of  sequences  where  m,  =  7. 


Table  1 

The  Probabilities  of  a  Specific  Sequence  of  n.  Digits  Occurring  Each 
Time  In  n  Trials  tor  an  Ideal  Binary  Process 


1 

2m, 

M 

Probability  That  the  m  Digits  Will  Be  Repeated  Each 

Time  in  n  Trials 

■ 

Number  of  Trials  (n) 

1 

■i 

2 

3 

4 

5 

6 

7 

8 

T 

2 

Di 

0.25 

0.125 

0.0625 

0.03125 

0.01563 

0.00761 

2 

4 

0.06250 

0.01563 

0.00391 

0.00098 

0.00024 

0.00006 

3 

8 

mm 

0.01563 

0.00195 

0.00024 

0.00003 

0.00000 

mu 

4 

16 

0.0825 

0.00391 

0.00024 

0.00002 

0.00000 

5 

32 

0.03 

0.00090 

0.00003 

0.00000 

■ 

7 

128 

0.01 

0.00010 

0.00000 

It  follows  that  in  choosing  a  value  of  m,  consideration  should  be  given  to  making  it  as 
near  to  7  as  possible.  Of  course,  the  complexity  of  testing  is  undoubtedly  also  directly 
dependent  upon  m.  A  balance  must  be  struck  between  vulnerability  and  complexity.  In 
devising  the  entire  test,  consideration  might  be  given  to  the  use  of  at  least  one  measure 
function  for  which  m  =  7,  and  this  should  be  a  direct  indicator  of  erroneous  accept  indica 
tions  from  repeated  sequences. 


APPUCATION  OF  THEORY 

We  are  ready  to  consider  an  example  to  illustrate  the  procedure  for  applying  the  theory 
developed  above.  A  rather  arbitrary  case  is  being  used  to  eliminate  the  need  for  special 
consideration  which  might  be  required  in  initially  setting  the  p -tolerances.  Thus,  instead 
of  starting  with  given  p-tolerances,  a  nominal  value  of  p^  =  0.5  was  chosen.  The  lower 
and  iqiper  limits,  ^pj  and  Ap,,  were  then  selected  Individually  from  a  shuffled  "deck"  of 
cards.  The  values  obtained  are 


^Pl  =  0.062  and  Apj  =  t  0.038. 

It  follows  that  the  p-tolerances  are  0.438,  the  lower  limit,  and  0.538,  the  upper  limit. 

We  will  examine  the  following  three  measure  groups  as  part  of  the  demonstration  of 
the  theory:  p,  6p*q*,  and  p^  +  4p3q.  Next  we  make  the  assumption  that  an  error  of  5 
percent  or  less  is  permissible.  We  find  from  Fig.  2  that  a  sample  size  n  of  about  1,000 
is  required.  We  note  that  condition  (26),  when  evaluated,  gives  ne^  =  0.125,  a  value  con¬ 
sidered  close  to  zero  and  satisfying  (35)  as  well.  We  should,  therefore,  expect  a  reasonable 
approximation  of  the  binomial  distri^tion  using  the  normal  law  equivalent. 
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In  Appendix  B  it  has  been  shown  that  a  good  indication  of  the  upper  bound  of  the  error 
in  can  be  obtained  from  Fig.  2,  and  that  the  error  in  p  can  be  within  this  bound.  In 
Appendix  B  the  three  specific  groupings  selected  for  our  discussion  have  been  evaluated 
with  regard  to  the  error  in  p.  For  p^  =  p,  the  error  in  p  is  Just  the  error  in  p^.  For 
Pg  =  6p*q*,  the  relative  error  in  phas  an  asymptote  at  p  =  0.5.  For  p.  =  p^  +  4p®q,  the 
asymptote  for  the  relative  error  in  p  occurs  at  p  =  1 .  Since  the  asymptote  for  p,  *  6p*q2 
occurs  at  p  =  0.5,  which  is  within  the  region  of  measuring  interest,  we  would  suspect  that 
that  particular  grouping  might  not  be  suitable.  This  turns  out  to  be  the  case  for  other 
reasons  as  well.  Because  this  function  has  some  interesting  qualities  it  will  be  treated 
further.  The  relative  error  in  p  for  Pg  =  p^  +  4p3q  is  less  than  the  relative  error  in 
pg  over  a  wide  range  of  values  of  p ,  including  the  region  of  interest  between  0.438  and 
0.538. 


Graphically  Determined  Acceptance  Characteristics 

Using  (38)  we  can  obtain  values  for  and  kj.  At  this  point  we  can  use  graphical 
techniques  in  obtaining  the  acceptance  characteristics.  By  selecting  at  least  two  values 
for  X,,  and  using  (41),  points  are  obtained  to  make  a  straight-line  estimate  of  the  acceptance 
characteristic  when  plotted  on  arithmetic  probability  graph  paper.  By  plotting  the 
tolerances  separately,  the  difference  indicated  by  (33)  can  be  obtained  gr^;>hically  from  the 
two  straight-line  plots  on  the  graph  paper  itself.  This  was  actually  done,  and  Figs.  9-12 
are  the  result.  To  illustrate  the  degree  of  agreement  possible,  points  on  the  acceptance 
characteristics  were  obtained  using  data  from  a  table  of  the  cumulative  binomial  function.*' 

In  Figs.  9,  10,  and  12,  the  straight  lines  are  adjudged  to  be  a  reasonable  connection  of  the 
triangularly  designated  points.  The  circularly  designated  points  were  obtained  from  the 
table  of  the  cumulative  binomial  function.  Because  the  measure  grotq)  6p’q^  (Fig.  11) 
exhibits  some  interesting  qualities  for  the  tolerances  which  were  obtained,  it  was  decided 
to  plot  the  composite  or  acceptance  characteristic  curve  by  calculating  a  number  of  points, 
indicated  by  crosses,  using  (33)  directly.  In  Fig.  11,  the  curve  which  has  been  drawn  was 
obtained  graphically  from  Fig.  10,  and  the  crosses  and  circles  are  points  obtained, 
respectively,  from  (33)  and  (32).  Further  evidence  of  the  agreement  obtainable  is  shown 
in  Fig.  12;  the  same  designations  for  points  applies  as  in  Fig.  11. 

We  note  from  Fig.  11  that  the  two  lines  corresponding  to  trte  distributions  about  the 
two  tolerances  are  both  incomplete  and  close  together.  I^e  terminal  point  of  the  two 
lines  corresponds  to  the  maximum  value  of 

P,  =  6p*q* 

which  occurs  for  p  =  0.5.  The  two  lines  are  close  together  because  p,  is  symmetrical 
about  p  =  0.5;  there  are  two  values  of  p  for  each  value  of  p,.  In  case  this  particular 
function  is  to  be  used  in  determining  values  of  p,  ambiguities  will  arise,  and  other  means 
have  to  be  used  to  resolve  the  ambiguity. 

With  the  aid  of  Figs.  3  and  4,  values  of  p,  can  be  transformed  into  values  of  p.  Then 
the  curves  of  Figs.  9, 11,  and  12  can  be  used  to  obtain  acceptance  characteristics  for  each  of 
the  measure  groups,  Figs.  13-15.  The  acceptance  characteristic  is  aplot,  on  linear  scales,  of 
the  probability  that  the  specific  measure  groups  occur  for  particular  values  of  p  when 
n  and  k  are  fixed.  Again,  to  indicate  the  effect  of  the  normal  law  approximation,  a  com¬ 
parison  is  made  with  results  obtained  using  the  table  of  the*  cumulative  binomial  probabil¬ 
ity  function. 


■(■The  staff  of  the  computational  laboratory,  Harvard  University,  op.  cit. 
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Fig.  9  -  Tolerance  lines  (acceptance 
characteristics)  for  the  measure 
function  M(438, 538, 1000, p).  The 
straight  lines  join  points  calculated 
by  Eq.(41).  Note  thatthe  points  cal¬ 
culated  by  the  cumulative  binomial 
function  lie  on  these  straight  lines 
also.  The  values  -4(Bki  -  H)  are: 
one  minus  the  left-hand  scale 
reading. 


Fig.  10  -  Tolerance  lines  (acceptance 
characteristics) for  the  measure  function 
■(363,372, 1000,6piqi) 


PROBABILITY  OF  ACCEPTANCE  FOR  M(22S,3r2.IOOO.P*-f 4p>4)-PERCENr 
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Figt  11  -  Graphical  composite  (solid 
curve)  of  tolerance  lines  shown  in 
Fig.  10,  The  values  calculated  by 
the  cumulative  binomial  function 
(circled  points)  are  shown  for 
comparison. 
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Fig.  12  -  Tolerance  lines  (acceptance 
characteristics  (for  the  measure  func¬ 
tion  M(225t372,1000tp4  +4p3q).  The 
values  for  -4  (x|||  -  h)  are:  one  minus 
the  left-hand  scale  reading. 
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Fig.  13-  Acceptance  characteristic  for  the 
measure  function  M(436t538,1000,p)  (solid 
curve)  obtained  graphically  from  Fig.  9. 
Comparison  with  three  decision  sequences, 
as  indicated. 


Fig.  14  -  Acceptance  characteristic  for 
the  measure  function m(363,372, 1000, 6p*q>  ) 
(solid  curve)  based  on  Eq.  (33).  Compar¬ 
ison  with  four  decision  sequences,  as 
indicated. 
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Fig.  15  -  Acceptance  characteristic  for  the  measure 
function  M (225, 372, lOOOfP^  -i-4pA|)  (solid  curve) 
obtained  graphically  from  Fig.  12.  Comparison  with 
three  decision  sequences,  as  indicated. 


Effect  on  Acceptance  Characteristics  of  Repeated  Measurements 
and  a  Decision  Criterion 

Having  obtained  the  acceptance  characteristics,  the  effect  of  using  an  acceptance 
decision  criterion,  such  as  that  given  by  (48),  can  be  determined.  The  acceptance 
characteristics  gives,  for  specific  values  of  p,  the  probability  of  acceptance  in  a  single 
trial  as  the  ordinate  in  Figs.  13-15.  Figure  8  is  a  plot  of  the  probability  that  there  will 
be  or  more  occurrences  of  specific  groups  of  digits  (the  oi^inate)  as  a  function  of  the 
probability  of  acceptance  in  a  single  trial  (the  abscissa).  We  can  use  graphical  means 
to  obtain  new  characteristics:  the  probability  that  there  will  be  or  more  acceptances 
in  n,  a  10  independent  trials.  To  show  what  such  characteristics  look  like  and  to  indicate 
the  variation  to  be  expected,  three  values  of  k^  (4,  5,  and  6)  were  selected.  The  results 
are  superimposed  on  the  acceptance  characteristics.  Figs.  13-15.  The  increase  in  sharp¬ 
ness  is  apparent.  In  Fig.  14,  four  values  of  k^  were  used  to  show  the  complete  trend  as 
k,,  is  increased. 

The  increase  in  sharpness  of  the  acceptance  characteristics  indicates  that  the 
theoretical  line  of  demarcation  between  the  accept  and  reject  regions  of  the  measure 
area  can  be  approached  quite  closely.  Thus,  conditions  (42)  can  be  satisfied  and  the 
effectiveness  given  by  (47)  should  be  of  high  order. 

From  Figs.  13-15  not  only  is  the  increased  sharpness  shown,  but  the  usefulness  of 
criterion  (49)  is  indicated. 

Conq)arison  of  Figs.  13,  14,  and  15  shows  that  two  of  the  functions  yield  similar 
acceptance  characteristics.  The  second,  p.  =  6p^q’,  however,  yields  a  different 
characteristic.  In  terms  of  (47)  it  can  be  seated  qualitatively  that  the  second  function  does 
not  offer  Very  good  acceptance  effectiveness  for  the  tolerances  chosen.  By  increasing 
both  tolerances,  the  tolerance  lines  become  separated  and  the  acceptance  characteristic 
can  be  made  to  reach  unity. 


Use  of  Graphical  Means  for  Adjusting  Tolerances 

As  a  demonstration  of  the  usefulness  of  the  graphical  method  in  the  adjustment  of 
tolerances,  we  consider  the  grouping  p^  =  6  p^q^  further.  With  reference  to  Fig.  16, 
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an  initial  change  in  tolerances  was  made  by  changing  k,  radically  to  265,  while  changing 
k,  only  slightly  to  375.  Using  (41),  calculations  yield  the  points  for  drawing  the  straight 
line,  indicated  by  the  triangles.  In  this  case  the  lower  tolerance  was  obtained,  using  (40), 
after  the  extreme  value  had  been  obtained  for  the  case  where  k,  =  375.  This  choice 
assured  no  crossover  of  the  tolerance  lines  (see  Fig.  16).  This  choice  of  tolerances  still 
gives  a  depressed  central  region  up  to  the  point  where  the  \4)per  limit  line  crosses  the 
line  of  maximum  value  of  p^,  0.375.  To  remove  the  depressed  region  it  is  necessary  to 
increase  the  iq)per  limit  further.  A  clue  to  the  amount  of  increase  is  obtained  from  Fig.  5 
(n  =  1000):  it  is  found  that  for  this  particular  function,  when  at  its  maximum  value, 
readings  (i.e.,  values  of  k)  up  to  about  430  are  still  quite  probable.  However,  an  arbitrary 
decision  was  made  to  accept  values  of  k  of  about  410  with  1  <  x,,  <  2.  At  this  point  it  was 
observed  from  the  plots  of  the  first  revised  tolerances  that  the  slope  of  the  two  lines  is 
nearly  the  same.  By  sketching  in  on  a  worksheet  the  new  line,  an  attempt  was  made  to 
find  the  lower  limit  gr^hically.  Here,  it  was  only  necessary  to  go  far  enough  with  the 
lower  limit  line  to  permit  a  small  amount  of  crossover.  Thus,  it  was  found  that  a  value 
of  about  310  could  be  used.  It  was  decided  to  compare  these  lines  with  the  lines  which 
would  be  obtained  for  other  comparable  values  -  hence,  the  lines  for  kj  =  302  and  k,  s 
400.  To  show  the  results  of  the  graphical  process,  points  for  each  of  the  lines  mentioned 
were  calculated  using  (41).  Note  that  in  the  case  of  the  last  two  upper  limit  lines  it  was 
necessary  to  plot  points  which  have  only  graphical  significance  since  they  are  beyond  the 
maximum  value  of  p^. 

To  illustrate  the  effect  of  choosing  the  various  tolerances  on  the  actual  acceptance 
characteristics,  two  of  the  sets  of  tolerances  referred  to  in  the  description  of  the  graphical 
process  have  been  plotted  (Fig.  17)  along  with  the  previously  obtained  acceptance  charac¬ 
teristic.  From  Fig.  17  it  can  be  seen  that  the  increase  in  acceptance  effectiveness  is 
obtained  at  a  sacrifice  in  rejection  effectiveness. 


Fig.  16  -  Tolerance  line  adjustments 
obtained  by  changing  k]  and  k]  for  the 
measure  function  M(k|,  kj.lOOO.bptqt ) 
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Fig.  17  -  Effect  of  choosing  various  tolerances  (values  of  ki  and  kj) 
on  the  acceptance  characteristic  of  M  (ki ,  k}tl000,6p>q>  ) 


Application  of  Graphical  Means  in  Comparing  Measure  Functions 

The  success  of  graphicsd  means  suggests  the  possibility  of  using  these  results  as  a 
rough  indication  of  acceptance  and  rejection  effectiveness.  Such  an  indicator  is  useful  in 
comparing  results  to  be  e]q;>ected  with  Ute  various  measure  functions.  From  Figs.  9,  10, 

12,  and  16  the  upper  limit  on  nearly  each  of  die  tolerance  lines  is  coincident  with  the 
calculated  point;  the  end  of  the  probability  graph  paper  yields  a  useful,  although  arbitrary, 
range  of  the  characteristic.  This  choice  offers  a  useful  standard  point  of  reference.  In 
this  kind  of  indication,  two  factors  are  of  significance:  (a)  die  value  of  the  measure  func> 
tion  at  the  point  of  intersection  of  the  tolerance  line  with  lAe  p-tolerance  value,  and  (b) 
the  range  of  values  of  p  out  to  the  significant  end  of  the  tolerance  line.  These  two  factors 
ouUine  standardised  portions  of  the  regions  r,  and  in  Fig.  7. 

Following  the  foregoing  method,  two  sets  of  values  are  given  in  Table  2:  those  for 
the  acceptance  region  (indicating  the  extent  to  which  desired  values  of  p  are  rejected) 
and  those  for  the  rejection  region  (indicating  the  extent  to  which  undesired  values  of  p  are 
accepted).  There  are  limitations  in  the  use  of  the  data  of  Table  2.  While  the  first  entry 
for  the  acceptance  region  is  straightforward,  the  second  entry  contains  a  dash.  This  dash 
must  be  interpreted  as  a  failure  cA  the  tolerance  line  to  reach  the  end  of  the  graph  paper. 
Another  difficulty  arises  because  there  is  a  depression  in  the  acceptance  characteristic 
within  the  acceptance  region.  This  situation  is  handled  by  introducing  die  middle  category. 
The  point-of-intersection  entry  in  the  middle  category  indicates  the  lowest  point  reached 
by  the  depressed  portion  of  the  characteristic;  in  die  cases  tabulated,  the  region  affected 
by  the  depression  in  the  acceptance  characteristic  is  included  between  die  two  p-tolerances. 
Aside  from  these  special  interpretations  die  data  can  be  used  in  comparing  different 
measure  functions.  From  the  tabulation  we  find  that  of  the  three  functions,  the  function 
M(225,372,l(X)0,p^  4p^q)  has  the  smallest  areas  r,  and  ■, ,  and  these  two  areas  are  of 

^ipT'oximately  the  same  size.  Hence,  this  function  most  nearly  satisfies  (42)  and  should 
give  the  best  overall  acceptance  characteristic. 


Vulnerability  of  Measure  Functions  to  Erroneous  Accept  Indications 

Having  obtained  a  rough  compariscm  of  the  three  measure  functions  being  used  in 
this  application,  it  is  necessary  to  make  some  comparison  (A  the  capability  for  each  of 
these  functions  to  reject  certain  known  repetitive  sequences.  The  sequences  have  been 
examined  in  Appendix  C. 
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In  Appendix  C  two  methods  of  sampling  sequences  made  up  of  repeated  groups  of  digits 
are  used.  The  results  of  the  investigation  of  Appendix  C  are  plotted  in  Fig.  18.  For  both 
sampling  methods  the  probability  of  acceptance  for  each  of  184  sequences  has  been  obtained. 
In  Fig.  18  this  probability  of  acceptance  for  each  of  the  184  sequences,  and  for  the  three 
measure  functions  being  considered  here  for  purposes  of  illustration,  has  been  plotted. 


Fig.  18  -  Effect  of  repeated  sequences  on  three  measure  functions 


Aside  from  the  fact  that  sampling  method  11  (see  Fig.  18)  yields  acceptance  indications 
from  more  sequences,  but  with  generally  lower  probabilities,  the  individual  vulnerability 
of  the  measure  functions  increases  in  the  following  order; 

m(438,538,1000,p) 

m(3 10,410, 1000,6p2q2) 

m(225, 372, 1000,  p^  +  4p3q). 

For  the  measure  functions  being  considered,  m  is  equal  to  1  in  the  first  function  and  to  4 
in  the  other  two  functions.  From  Fig.  18,  the  first  erroneous  indications  for  the  more 
stringent  method  n  occur  respectively  attn,=  2,  3,  and  5,  the  measure  functions  being  con¬ 
sidered  individually.  If,  however,  the  third  measure  functions  is  combined  with  either  of 
the  first  two  measure  functions,  the  only  common  region  of  erroneous  acceptance  indication 
occurs  for  ni,=  6.  This  represents  some  increase  in  the  fidelity  of  go/no-go  accept 
indications. 

From  Fig.  18  we  see  that  for  the  case  in  which  two  measure  functions  are  combined, 
there  are  seven  sequences  which  will  cause  erroneous  accept  indications.  A  further 
increase  in  the  fidelity  of  accept  indications  should  be  obtained  by  determining  the  measure 
function  which  includes  just  the  7  sequences  which  cause  the  erroneous  accept  indications. 
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For  the  illustrative  example  being  considered,  a  satisfactory  go/no-go  measuring 
system  consists  in  using  three  measure  functions,  m(438, 538, 1000, p),  M(310,410,1000,6p^q^, 
and  m(225, 372, 1000, p^  +  4p^c4,  on  which  the  limits  and  acceptance  characteristic  have  not 
been  determined  but  which  includes  the  seven  sequences  which  cause  the  erroneous  accept 
indications. 


SUMMARY  AND  CONCLUSIONS 

The  general  problem  of  testing  a  particular  process  for  randomness  reduces  to  the 
problem  of  examining  a  sequence  of  numbers  to  determine  that  the  set  of  numbers  have 
statistical  characteristics  which  are  within  predictable  limits.  The  sequence  or  set  of 
numbers  can  be  arranged  into  an  n  by  m  array.  The  dimensions  of  the  array  yield  certain 
information  about  the  tests  which  might  be  performed  to  determine  the  randomness  of  the 
numbers  contained  in  the  array.  The  length  n  (number  of  rows)  of  the  array  sets  the  upper 
bound  on  the  measuring  error  (in  the  value  of  p),  whereas  the  width  m  (number  of  columns) 
is  an  indicator  of  the  digital  groupings  which  can  be  used  for  randomness  testing.  The 
width  m  of  the  array  serves  also  (for  go/no-go  tests)  as  an  indicator  of  the  capability  of 
specific  digital  groups  to  reject  nonrandom  sequences  (i.e.,  groups  of  digits  which  are 
repeated)  and  thereby  to  prevent  false  (go)  indications. 

Consider  the  case  of  an  array  of  binary  digits.  If  the  assumptions  are  made  that  each 
entry  in  the  array  is  independent  of  any  other  entry,  and  the  probabilities  of  generating 
the  two  digits  remains  constant,  then  the  array  could  have  been  generated  from  Bernoulli 
trials.  Thus  the  binomial  distribution  and  its  characteristics  serve  as  the  theoretical 
basis  of  comparison  for  observed  results. 

It  has  been  shown  that  there  are  many  kinds  of  tests  which  can  be  devised  to  measure 
randomness.  Essentially  these  are  tests  in  which  p  is  measured  indirectly,  by  using  any 
term  or  combination  of  terms  in  the  binomial  ejqiansions  [p  (1  -  p)}”*  In  selecting  a 
particular  value  for  m,  an  effective  transformation  in  the  array  is  made,  from  a  n-column 
array  to  a  single  column  array,  and  this  is  tantamount  to  a  change  in  number  system  of 
higher  order  radix;  the  original  number  system  radix  is  2.  Thus  the  selection  ofais 
made  partly  on  the  basis  of  the  digital  group  or  groups  of  digits  considered  of  interest. 

Although  exhaustive  testing  for  randomness  is  indicated,  this  investigation  has  con¬ 
sidered  the  results  to  be  expected  from  single  tests  and  for  tests  repeated  a  small  number 
of  times.  The  investigation  was  directed  to  automatic  tests,  with  go/no-go  indication. 
Because  the  binomial  distribution  is  not  as  well  tabulated  as  is  the  normal  distribution, 
the  theory  was  developed  about  the  latter. 

It  has  also  been  shown,  particularly  for  the  go/no-go  measuring  case,  that  the  choice 
of  the  value  of  m  depends  upon  the  desired  capability  for  the  test  to  reject  repeated  sequences 
of  digits.  A  value,  m  =  5  or  6,  satisfies  this  requirement  if  the  two  means  used  to  generate 
repeated  sequences  are  the  only  ones  to  be  considered.  The  mere  choice  of  a  value  for  m 
does  not  prevent  repeated  sequences,  when  they  occur,  from  causing  false  "go”  indications. 
Once  m  has  been  selected,  various  digital  groups  can  be  tested  to  determine  which  repeated 
sequences  are  likely  to  cause  the  false  indications.  It  should  be  noted  that  m  should  be 
given  as  large  a  value  as  possible,  and  if  it  is  an  odd  number,  the  resulting  effect  of 
repeated  sequences  should  be  minimized.  With  m  an  odd  number,  repeated  sequences 
which  might  cause  false  indications  should  be  irregular;  hence,  it  may  be  easier  to  devise 
tests  to  prevent  such  false  indications.  Thus  combinations  of  tests  are  implied  to  cope 
with  the  problem  of  false  indications.  On  the  other  hand,  the  method  of  generating  the 
random  digits,  which  essentially  is  being  tested,  might  profitably  be  examined  to  determine 
if  particular  sequences  are  likely  to  be  generate.  A  particular  generator  may  be  prone 
to  fall  into  a  particular  pattern  or  sequence  of  digits,  and  this  pattern,  if  known,  serves 
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to  evaluate  the  need  for  concern  about  repeated  sequences  when  devising  the  tests.  Indeed, 
it  may  be  impossible  for  some  generators  to  change  from  a  random  to  a  nonrandom 
(repeated  sequence)  state  without  the  test  detecting  the  transition.  While  there  is  no 
unique  solution  to  the  problem  of  selecting  the  value  of  m,  these  are  the  considerations 
which  must  be  given.  Once  the  selection  of  m  has  been  made,  the  possible  digital  groups 
made  available  for  the  tests  should  be  checked  to  determine  their  effectiveness  in  measur¬ 
ing  p;  an  error  analysis  in  accordance  with  Appendix  B  is  required  to  avoid  gross  errors. 
Means  are  given  for  determining  the  upper  bound  on  the  error  to  be  expected  in  measur¬ 
ing  p.  While  this  error  is  primarily  determined  when  the  length  of  the  array  of  the  sample 
size  (n)  is  selected,  the  error  analysis  for  particular  digital  groups  will  show  what 
departures  from  the  upper  bound  of  error  can  be  expected. 

Having  determined  both  the  particular  digital  group(s)  and  n,  with  known  limits  for 
P,  the  tolerances  and  k^  on  the  group  count  can  be  assigned.  The  measure  function 
thus  obtained  is  of  the  formM(ki,kj,n,p,).  The  effectiveness  of  particular  measure  func¬ 
tions  can  be  determined  qualitatively  once  the  acceptance  characteristic  is  obtained.  An 
acceptance  characteristic  is  a  graph  of  M(ki,k2,n,p^)  vs  p.  Any  acceptance  limit  line  can 
be  compared  with  the  ideal,  a  straight  line  of  infinite  slope.  In  practice,  there  will  be  a 
small  number  of  measure  functions  to  be  compared,  and  the  best  measure  function  will  be 
that  having  the  smallest  dispersion  of  both  limit  lines. 

The  acceptance  characteristic  is  essentially  composed  of  two  normal  distributions, 
corresponding  to  the  upper  and  lower  limits  of  p.  Because  of  this  fact,  acceptance  charac¬ 
teristics  are  readily  obtained  with  the  aid  of  tables  of  the  normal  distribution  function. 

With  the  aid  of  normal  probability  gr^h  paper  it  is  possible  to  obtain  the  characteristic 
graphically. 

The  acceptance  of  the  measure  function  establishes  the  results  which  can  be  expected 
in  a  single  go/no-go  test.  The  acceptance  characteristic  of  any  measure  function  can  be 
made  to  approximate  the  ideal  more  nearly  by  using  a  relatively  small  number  of  tests 
and  adopting  a  decision  criterion.  It  is  shown  that  considerable  improvement  is  achieved 
if  a  sequence  of  10  tests  is  used,  with  a  decision  to  accept  based  on  5  acceptances  out 
of  10. 

It  has  been  shown  that  dynamic,  automatic  testing  of  randomness  can  be  achieved 
with  useful  results.  The  requirement  fora  relatively  small  digital  group,  to  avoid  false 
"go”  indications,  and  for  a  relatively  small  number  of  repeated  tests  suggests  that 
practical  devices  can  be  built  to  perform  the  tests. 

In  developing  the  theory,  a  means  for  measuring  p  more  directly  was  described. 

This  alternative  method  consists  of  obtaining  statistically  suitable  samples,  each  of  size 
n,  of  the  frequency  of  occurrence  (or  count)  of  specific  groups  of  digits.  If  enough  samples 
are  obtained,  the  parameters  of  the  distribution  of  count  can  be  calculated.  It  can  there¬ 
fore  be  determined  statistically  that  the  counts  are  distributed  randomly.  Furthermore, 
from  the  average  value  of  the  count  the  most  probable  value  of  the  grovq)  frequency  of 
occurrence  can  be  calculated.  The  value  of  p  can  thus  be  measured.  By  the  application 
of  this  method,  an  independent  check  is  available  for  verifying  the  results  of  go/no-go 
tests.  The  advantage  of  this  alternative  method  lies  in  the  fact  that  the  instrumentation 
requirements  may  be  somewhat  less  than  for  the  go/no-go  method. 


APPENDIX  A 


SOME  RELATIONS  BETWEEN  EXPRESSIONS 
FOR  THE  NORMAL  DISTRIBUTION  FUNCTIOK 


Given  the  normal  density  function 
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the  normal  distribution  function  is  defined  as 


♦(X)  = 
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*)  dt. 
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Furthermore, 


J  ‘‘t  =  — ^  J  *’^**'*^ 

J., 


dt  *  1. 
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We  can  divide  (A3)  into  three  parts: 


j  d>(t)  dt  +  j  dt  +  j  0(t) 


dt  =  1. 


(A4) 


Since  we  are  dealing  with  the  normal  density  functimi,  which  is  symmetrical  about  t  =  0, 


j  d>(t)  dt  =  J  ^(t)  dt. 


(A5) 


It  follows  that 


f  ^t)  dt  ♦  f  ^t)  dt  =  2  f  <p{t)  dt 

■I 


=  2  1  0(1)  dt. 


(A6) 


(A7) 
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Using  this  result  In  (A4)  we  obtain 


<^(t)  dt. 


(A8) 


Let 


4(x)  = 


(^(t)  dt. 


(A9) 


Then 


♦(X)  = 


J  '4>(t)  dt  + 


I  d>(t)dt. 


(AlO) 


Also, 


♦(-X) 


4>(t)  dt. 


(All) 


Now 


♦(x)  +  ♦(“X) 


f  0(1)  dt  +  f  0(t)  dt  +  f 


0(t)  dt. 


Using  (AS) 


♦(X)  + 


♦(-X)  =  f  0(t)  dt  +  I  0(t)  dt  +  J 


0(t)  dt  =  1. 


Therefore, 


♦(-X)  =  I  -  Bfx). 

Now  take  the  difference  between  (AlO)  and  (All): 


♦(X) 


-  4(-x)  =  j  0(t)  dt  +  J  0(t)  dt  -  J  0(t)  dt 


f  0(t)  dt  -  f 

••• 


(A12) 


0(t)  dt  (using  (A9)). 
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Therefore, 


i(x) 


-4(-x)=  J 


dt. 


(A13) 


But,  using  (A12), 


4(x)  -  [1 


-  4>(x)]  =  J 

•* 


<t>(t)  dt 


or 


24(x) 


-1=1  dt 


or 


“f 


■tCx)  =  K  +  M  <t>(t)  dt 


(A14) 


Also, 


♦(-X)  =  M  -  H  I 


<#!(t)dt. 


(A15) 


Many  sets  of  tables  of  the  normal  distribution  function  are  available.  A  rather  com¬ 
plete  one  (“Tables  of  the  Probability  Functions,’'  Vol.  II,  FWA,  WPA  for  the  City  of 
New  York,  1942)  tabulates  the  function 


J  '^(t)  dt. 


APPENDIX  B 


THE  RELATION  BETWEEN  THE  ERRORS  IN  p,  AND  p. 


The  ability  to  select  an  upper  bound  to  the  error  expected  in  making  a  measurement 
is  of  considerable  significance.  Equation  (19)  provides  a  mean  for  selecting  an  upper 
bound  to  the  error  in  measuring  p,,  the  probability  of  occurrence  of  a  specific  grouping 
of  digits  whose  functional  relationship  may  be  one  or  more  of  the  terms  in  (12).  Use  of 
p  ,  in  setting  the  upper  bound  of  error  provides  a  general  way  of  setting  the  error,  but 
it  does  not  show  how  this  error  affects  the  value  of  p,  the  independent  variable  in  func¬ 
tions  represented  by  p^. 

It  has  been  shown  that  there  are  a  large  number  of  possible  functions  of  p  which  can 
be  used  to  define  p,.  We  will  consider  a  small  number  here  to  show  the  relationship 
between  the  error  in  p^,  which  can  be  selected  by  (19),  and  the  corresponding  error  in  p. 

In  general, 


p,  =  f(p)-  (®1) 

Let  us  differentiate  (Bl): 

pi  =  f(p).  (B2) 

dp 


Now,  suppose  we  let  the  differentials  dp^  and  dp  approximate  the  respective  differences 
Ap|  and  Ap.  Substituting  the  differences  for  the  differentials  in  (B2)  we  obtain 

^P,  ~  t'(P)  (B3) 


where 


Ap,  =  « 


(B4) 


the  small  number  of  (19). 

In  (B3)  we  have  the  (approximate)  relation  between  the  error  in  p,  and  the  corre¬ 
sponding  error  in  p.  For  purposes  of  comparison,  we  are  interested  in  the  relative  errors. 
We  obtain  approxinoations  of  the  relative  errors  as  follows: 


P. 


which,  upon  rewriting,  becomes 


1 

fV) 


(B5) 


(B6) 
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To  illustrate  the  use  of  (B6)  let  us  consider  the  following  seven  measure  functions 
whose  relative  errors  are  as  tabulated. 


Relative  Error  Ratio 

Measure 

/Ap\  j 

Location  of 

Function 

Asymptote 

(P.) 

\p// Vp,/ 

(P) 

p 

1 

none 

2pq 

1  -  p 

1  -2p 

0.50 

6p*q^ 

1  “  P 

0.50 

2(1  -  2p) 

3p*q 

1  -  p 

2  -  3p 

0.66 

4p^q 

1  -  P 

3  -  4p 

0.75 

Sp^q 

1  -  p 

4-5p 

0.80 

p^  +  4p3q 

4  -  3p 

12(1  -  p) 

1.00 

To  aid  in  comparing  the  above  tabulated  functions,  Fig.  B1  has  been  drawn  and  shows 
how  the  relative  error  ratio  (Ap/p)/(Ap,/p,)  varies  with  p.  Of  particular  interest  are 
those  functions  for  which  |Ap/p  |/|Ap,/p,|  <  1,  representing  direct  utility  of  the  selected 
value  of  e  as  an  upper  bound  of  error. 

The  above  tabulated  functions  are  a  sample  of  the  possible  functions  which  could  be 
assigned  to  p^.  Once  a  particular  functional  relation  is  chosen  for  use  in  making  meas¬ 
urements,  an  error  analysis  should  serve  to  indicate  the  maximum  error  likely  to  be 
encountered  within  specific  ranges  of  numerical  values  of  p.  From  Fig.  B1  the  location 
of  the  asymptote,  if  one  exists  for  a  specific  function,  is  a  useful  indicator  of  the  region 
of  gross  errors.  Because  of  this,  the  location  of  the  asymptote  is  tabulated  above  for 
each  of  the  functions. 

It  can  be  concluded  that  an  error  analysis  can  indicate  the  utility  of  a  specific  func¬ 
tion  in  making  measurements  of  p.  Furthermore,  for  certain  ranges  of  values  of  p, 
functions  can  be  chosen  such  that  |Ap/p|  <  1,  and  the  upper  bound  determined  on  the  basis 
of  Pg  remains  an  upper  bound  for  errors  in  p. 


approx  relative  error  ratio  (NUMERIC) 
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0  02  04  06  08  10 

PROBABILITY  THAT  A  ONE  IS  GENERATED  (NUMERIC) 


Fig.  B1  -  The  dependence  of  the  rela¬ 
tive  error  ratio  in  measuring  p  upon 
p  for  seven  measure  groupings  or 
number  symbols 


APPENDIX  C 


REPETITIVE  SEQUENCES  AND  THEIR  EFFECT 
ON  GO/NO-GO  TESTING  OF  RANDOMNESS 

Suppose  that  a  sequence  of  digits  is  obtained  by  repeating  a  group  of  fixed  digits. 

If  m,  is  the  number  of  binary  digits  which  form  the  group  of  fixed  digits,  then  a  sequence 
of  any  length  can  be  obtained  merely  by  repeating  the  fixed  group.  The  sequence  thus 
obtained  can  be  arranged  in  the  form  of  an  n  by  m  array.  The  content  obtained  for  the 
array  will  depend  upon  the  way  in  which  the  sequence  is  sampled  to  derive  entries  for 
the  array.  Consider  two  ways  of  sampling  (there  are  many  ways  which  could  be  devised). 

In  one  sampling  method,  each  digit  of  the  sequence  is  used  as  it  appears  in  the  sequence 
and  m-digit  groups  are  formed  from  the  sequence  until  there  are  n  groups.  In  another 
method,  the  sequence  is  sampled  to  obtain  m  consecutive  digits  starting  at  arbitrary  or 
essentially  random  places  in  the  sequence,  again  until  there  are  n  groups. 

To  show  how  the  array  content  can  be  derived,  it  is  necessary  t'>  know  what  measure 
groups  are  to  be  used  (i.e.,  p,  must  be  specified),  as  well  as  the  count  limits.  With  small 
values  of  m.,  and  for  small  vsuues  of  m,  only  a  small  number  of  entries  need  be  made  In 
the  array  because  these  entries  are  repeated.  Once  this  repeated  portion  has  been  obtained 
it  is  easy  enough  to  calculate  the  content  of  an  array  of  any  size. 

The  first  sampling  method  has  been  used  to  investigate  the  following  measure  func¬ 
tions  for  the  case  where  p  =  0.5: 

M(4S,5S,100,p),  M(3S,65,100,p),  (">=1) 

M(45,S5,100,2pq>,  11(35, 65, 100,2pq),  ( m  »  2) 

M(32,43,100,3p  3q),  M(23,52,100,3p>ci),  ( m  =  3) 

M(21,29,100,4p3q),  M(12,38,100,4p®q),  ( n>  =  4) 

M(12,19, 100,5 p4q),  M(S,26,100,5p*q),  (  n.  =  5). 

la  each  case  the  count  limits,  and  k,,  were  obtained  from  Fig.  5  at  the  appropriate 
values  of  p,  using  first  =  i  1  and  then  =  i  3.  The  results  are  given  in  Tables  Cl 
through  C4.  Included  in  Tables  Cl  through  C4  are  the  repeated  portions  of  each  sequence 
tried,  i.e.,  the  actual  digital  content  of  a  repetition  period.  From  the  sequence  of  repeated 
groups  of  digits  the  array  content  is  derived  as  described  above;  only  that  portion  of  the 
array  which  is  repeated  is  shown.  Repeated  sequences  are  located  together  whenever, 
for  a  given  value  of  m,,  the  array  contains  the  same  m-digit  groups,  regardless  of  their 
order  of  occurrence.  The  sequences  are  tabulated,  for  each  value  of  m,  for  increasing 
values  of  m,.  As  a  matter  of  interest  the  number  of  possible  arrangements  of  digits  for 
each  value  of  m ,  and  tn  is  given,  i.e.,  the  number  of  combinations  of  m,  things  taken  m  at 
a  time  or  of  m  things  taken  m,  at  a  time:  C^)  when  m,  >  m,  or  (iT)  when  m  >  m,. 

A  given  sequence  is  only  considered  once;  for  example,  all  zeros  or  ill  ones  are  con¬ 
sidered  once  for  each  value  of  m.  The  group  count  is  obtained  by  repeating  the  portion 
of  the  array  which  is  shown  an  appropriate  number  of  times,  as  determined  from  the 
value  of  n  indicated  for  the  particular  measure  function.  It  should  be  noted  that  when 
is  greater  than  m,  the  array  content  is  readily  obtained  by  taking  the  first  m  digits  from 
each  of  the  similar  groups  of  repeated  sequences  given  in  the  second  column.  Thus,  for 
example.  Table  C4  might  be  used  to  investigate  the  effect  of  a  value  of  m,  =  7  with  the 
measure  functions  given  in  Table  C9. 
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d  Accept  (A)  or  Reject  (R) 

Sampling  Methods 

Sequence  Sampling  Method  n 

Jt 

M 

h(455,545,1(XX>,p) 

R  (all) 

1 

■< 

1 

i; 

lA 

S 

lA 

lA 

S 

H 

A  (all) 

as 

1 

<< 

Group  Count 
(Numeric) 
(Total  varia¬ 
tion  of  +  45 
allowed) 

1 

1000 

455  -  500 

-H 

1 

1000 

621  -  711 

455  -  545 

O 

1000 

455  -  545 

Meaaore  Functioii  Eqnatkm,  Croup  Count,  an 
Indication  for  two  Sequence 

Sequence  Sampling  Method  I 

M 

M 

M 

'S' 

§ 

«A 

R0 

«A 

Q, 

m 

as 

•< 

1 

M 

§ 

<A 

R0 

«A 

$ 

at 

< 

as 

•< 

M 

li 

M 

M 

8 

S 

i 

m 

OS 

•< 

i 

§ 

S 

lA 

s 

a 

at 

< 

at 

< 

1  a  i 

ilU 

O 

1 

001 

s 

_ 

d 

n 

a 

H 

lA 

d- 

N 

V 

a 

o 

o 

m 

I 

o 

1 

n 

1 

t 

rt 

H 

m 

a 

f« 

H 

E 

1 

100 

S 

s 

o 

100 

s 

Portion  of 
Array  Which 

la  Repaated 
for  the 
Selected 
Groups 

_ 

1 

1 

8 

- 

1 

00 

10 

01 

*a4  e 

i-«  o  ^ 

00 

01 

00 

II 

00 

!S  O 

O 

11 

10 

10 

II 

Number  of 

ill- 

1 

s 

M 

* 

cs 

4 

* 

fS 

1 

• 

V 

PM 

Number  ofDigita 

Per 

Repetition  Period 

C.) 

fS 

r> 

1 

Digits  in  One 
Repetition  Period 

o 

o  ^ 

d 

01... 

10... 

001... 

010... 

100... 

d  wj  *4 
wwe  ^ 

f>4  «>M  e 

is  8l 

0011... 

1100... 

0110... 

1001... 

1110... 

1011... 

1101... 

0111... 

1 

Number 

1 

1 

•9  ^ 

1 

5 

6 

7 

■ 

M  ^ 

pM  pM  pM  w 

r  lA  EO 

M  pM  pN 

17 

18 

38 

*The  second  number  indicates  the  number  of  arrangements  previously  considered. 
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(page  42  Blanh) 


Hi 

III! 

■< 

i;l% 

%d  kv 
oo  u 

K 

et 

« 

755  -  845 

e 

m 

r- 

M 

1 

s 

M 

e 

44 

»• 

1 

ss 

p 

< 

m 

ft 

3 

o 

a 

o 

6 

o  ^  o 

^  O 

s  P  s  o 
e  ^  e  ^ 

— 

«H  md  ^ 

O  ^  ^ 

o  ^  ^  ^ 

■ 

0011 

0001 

0100 

0101 

1000 

Olio 

0001 

0010 

1000 

1010 

IIIO 

0011 

1000 

:8S 

523 

m 

n 

■ 

I 

0110 

1001 

1010 

■ 

■ 

0101 

1101 

0111 

■ 

■ 

HOT 

OUT 

0101 

0111 

1101 

1111 

1011 

1110 

1111 

* 

e 

t 

s 

1 

#1 

- 

p 

2 

10110... 

10101... 

01101... 

01011... 

*<4  *4  ^  O 

^  ^  o 

•■4  ^  O 

^  o  ^  ^ 

O  OM  M 

m 

110000... 

000101... 

010001... 

010100... 

100001... 

000110... 

011000... 

001010... 

100010... 

101000... 

000111... 

011100... 

110001... 

b'g 

4  0  0 

4  fi  fR 

2  O 

9  <N  «4 

001011... 

101100... 

110010... 

001101... 

010011... 

110100... 

010110... 

011001... 

100101... 

011010... 

100110... 

101001... 

001111... 

110011... 

111100... 

010111... 

011101... 

110101... 

011110... 

100111... 

111001... 

101011... 

101110... 

111010... 

011111... 

110111... 

111101... 

101111... 

111011... 

111110... 

H 

■HR 

mm 

3  S8S 

Hi 

n 

i 

■ 

C»  PP 
t-  *»*r* 

80 

81 

82 

B 

ses 

s 

B 

95 

98 

97 

1 

101 

108 

109 

104 

105 

106 

♦The  second  number  indicMee  the  number  ol  arrangements  preriously  considered. 
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51  011110...  11001 

52  110011...  11100 

SS  111001...  11110  .  , 

04  111100...  01111  R _ R _ B 

*Tke  eecood  number  Indicntae  the  number  of  nrrnngementa  previously  considered. 
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The  results  obtained  using  method  I  and  shown  in  Tables  Cl  through  C4  are  sum¬ 
marized  in  Table  C5.  Hie  purpose  in  choosing  two  measure  functions  for  each  value  of 
m  was  to  demonstrate  that  different  count  limits  (kj  and  kj)  can  have  somewhat  different 
repeated  sequence  effects.  While  the  difference  can  be  clearly  seen  from  Table  C5  (e.g., 
the  measure  functions  for  m  =  3  and  4),  such  a  drastic  change  in  limits  would  not  be  recom¬ 
mended  without  ascertaining  the  change  in  acceptance  characteristic  and  the  associated 
change  in  the  incidence  of  reject  indications.  This  trivial  example  serves  to  illustrate 
how  far  it  may  be  necessary  to  alter  the  limits  in  some  cases  to  achieve  a  significant 
change  in  accept  indication  in  the  presence  of  repeated  sequences. 


V 


Table  C5 

Repeated  Sequences  Which  Cause  Accept  Indications  for  the  Measure  Functions  Being  Tested 


Sequence  Sampling 
Method 

Measure  Function 

1 

ffl 

a 

Repeated  Sequence  Numbers 
Which  Give  Accept  Indications 
(from  Tables  Cl  through  C4) 

Approximate  Probability 
of  Accept  Indication 
(Numeric) 

I 

h(45,SS,100,p) 

D 

2 

3,4 

1 

M  (35, 65, 100, p) 

1 

n 

M  (4  55,545, i000,p) 

1 

I 

M(45,55,100,2pq) 

2 

D 

11  through  14 

H  (35,65, 100, 2pq) 

■ 

19  through  22 

n 

M(455,S45,1000,2pq) 

■ 

I 

M(32,43,100,3p3q) 

3 

5 

28  through  32 

M(23,52,100,3p>q) 

2 

3,4 

4 

15  through  18 

5 

28  through  32 

38  through  42 

n 

M(325,425,1000,3p2q) 

_ 

0.93 

I 

M(21,29,100,4o)q) 

1 

B 

135  through  141 

171  through  177 

1 

M  (12,38,100, 4p3q) 

D 

71  through  82 

1 

1 

135  through  141 

149  through  155 

171  through  177 

II 

M(208,292,1000,4p3q) 

1 

D 

71  through  82 

0.003 

1 

1 

135  through  141 

171  through  177 

0.74 

I 

M(12,19,100,5p<q) 

5 

lO 

178  through  184 

1 

m(5,26,  I00,5p«q) 

n 

M  (120, 190,1000, Sp*q) 

■ 

0.97 

Next  let  us  consider  method  II.  Now  we  are  concerned  with  a  sequence  which  is  made 
up,  as  before,  with  a  group  of  fixed  digits  m,,  which  are  repeated.  In  sampling  a  sequence, 
arbitrary  or  (assumed)  random  places  are  taken  to  select  an  entry  of  m  digits  for  making 
up  the  array.  To  illustrate  the  process  we  will  consider  the  following  measure  functions: 

M(455,545,1000,p) 

M(455,545,1000,2pq) 

M(325,425,1000,3p^) 


M(208,292,1000,4p3q) 

M(120,190,1000,5p^q). 
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For  each  sequence  to  be  tried,  the  array  content  is  determined  as  in  the  case  of  the  first 
sampling  method;  hence,  the  entries  of  Tables  Cl  through  C4,  which  include  the  portions 
of  the  arrays  which  are  repeated,  are  applicable  in  this  sampling  method.  In  this  case  it 
is  necessary  to  assign  (in  practice,  to  determine)  the  variation  in  count  which  is  likely 
to  occur;  for  our  example,  a  random  selection  of  entries  in  the  repeated  portion  of  the 
array  has  been  assumed.  It  is  natural  to  assume  that  in  the  long  ran  a  normal  distribu¬ 
tion  of  entries  should  be  obtained.  It  will  be  noted  that  the  limits  for  each  of  the  measure 
functions  have  been  obtained  from  Fig.  6,  at  the  appropriate  value  of  p,  (corresponding  to 
p  =  0.5),  for  =  1  3.  This  assignment  of  limits  permits  a  nearly  normal  variation  in 
count  to  be  obtained  under  normal  circumstances;  the  range  of  this  distribution  is  6a.  For 
purposes  of  illustration,  the  expected  count  is  assumed  to  have  the  same  type  of  distri¬ 
bution  -  a  normal  distribution  with  about  the  same  value  of  a.  Therefore  the  value  of 
total  variation  is  taken  to  be  ±3  a,  where 


a  - 


(Cl) 


and  kj  and  k,  are,  respectively,  the  lower  and  upper  limits  of  count  taken  from  the  meas¬ 
ure  function. 

In  Tables  Cl  through  C4  each  of  the  above  measure  functions  is  investigated.  For 
each  function  the  total  variation  (  ±  3<r)  is  given,  using  the  value  of  a  obtained  from  (Cl). 
Where  appropriate,  the  total  variation  is  included  in  the  group  count.  The  expected  group 
count  is  compared  with  the  measure  function  limits  (ki,k2)  and  the  expected  indication  is 
noted  (r  =  reject,  or  A  =  accept).  In  method  n,  when  the  group  count  is  compared  with 
the  measure  function  limits,  it  is  found  in  some  cases  that  only  part  of  the  expected  count 
falls  within  the  accept  limits.  In  these  cases  it  is  necessary  to  determine  the  approxi¬ 
mate  probability  of  getting  an  accept  indication.  The  example  we  have  chosen,  which  does 
not  give  tolerances  on  p,  is  assumed  to  have  an  acceptance  characteristic  which  is  the 
normal  curve.  With  this  assumption,  the  probability  of  acceptance  can  be  readily  deter¬ 
mined;  this  calculated  value  is  given  in  the  tables. 

For  purposes  of  comparison.  Table  C5  includes  the  accept  indications  obtained  for 
both  sequence  sampling  methods.  The  results  shown  by  Table  C5  are  indicative  of  what 
can  be  expected  from  an  investigation  into  the  effect  of  repeated  sequences  on  a  go/no-go 
test  device.  The  measure  functions  with  the  wider,  more  realistically  set  limits  are  more 
vulnerable  to  sampling  by  method  I. 

The  foregoing  discussion  serves  to  point  out  the  nature  of  the  problem  which  arises 
with  repeated  sequences.  The  examples  used  are  not  realistic,  but  they  have  served  to 
illustrate  some  of  the  conclusions  which  can  be  drawn.  It  is  appropriate  to  consider  the 
measure  functions  used  in  the  latter  part  of  the  report,  as  possibly,  a  more  realistic 
demonstration. 

The  three  measure  functions  being  considered  now  are  the  following: 

11(438,538, 1000, p) 

and  ll(310,410,1000,6p*q») 

M(225,372,1000,p^+4p3q). 

For  each  of  these  measure  functions,  the  acceptance  curve  is  available.  Figs.  13,  14,  and 
15,  respectively.  We  will  consider  each  of  these  functions  and  show  the  reaction  to 
repeated  sequences  when  sampled  by  both  methods  I  and  II. 
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Let  us  consider  the  formation  of  arrays  in  which  m  =  4  for  each  of  these  functions. 

This  poses  a  problem  insofar  as  the  first  of  the  three  functions  is  concerned;  for  purposes 
of  illustration  it  is  considered  that  a  single  column  sum  (one  of  the  four  obtainable  from 
the  4-digit  array)  will  be  used.  It  is  interesting  to  note  that  a  simplification  in  tabulation 
is  achieved  if  the  hexadecimal  number  system  is  used  to  write  the  array  entries.  The 
conversions  for  the  sequences  of  Table  C4  are  given  in  Table  C6.  Table  C7  includes  the 
hexadecimal  numbers  used.  In  Table  C8  are  shown  the  binary  and  hexadecimal  digits  of 
some  specific  groupings. 

Having  set  up  the  procedure  for  obtaining  arrays,  use  can  be  nuuie  of  the  sequences 
listed  in  Table  C4,  the  most  extensive  of  the  tables,  to  test  the  measure  functions  just 
listed.  Using  both  methods  of  sequence  sampling,  the  effect  of  each  of  the  sequences  can 
be  obtained;  however,  it  is  first  necessary  to  allow  a  variation  in  count  for  method  II,  as 
was  done  previously.  For  our  purposes,  a  count  variation  of  t  50  is  considered  suitable, 
since  this  value  is  very  nearly  obtained  from  Fig.  5  ( n  =  1000)  for  any  of  the  limits  used. 
The  results  are  given  in  Table  C9;  a  summary  of  these  results  is  given  in  Table  CIO. 

From  the  summary.  Table  CIO,  we  find  that  out  of  the  184  sequences  for  sampling 
method  I  there  are  24  accept  indications  obtained  from  use  of  the  first  two  measure  func¬ 
tions,  and  only  13  with  the  third.  In  the  case  of  sampling  by  method  II,  the  first  two  meas¬ 
ure  functions  give  83  and  59  accept  indications  respectively;  the  third,  only  23.  Of  the 
first  two  measure  functions,  the  second  gives  values  of  the  probability  of  acceptance  which 
are  lower,  for  sampling  by  method  n. 

It  should  be  noted  that  the  conclusion  previously  obtained  regarding  the  better  ability 
of  Method  I  to  make  tests  is  not  substantiated  in  Table  CIO.  The  inability  to  verify  a 
conclusion  merely  points  up  the  inability  to  specify  a  general  or  absolute  method  of  test¬ 
ing  for  the  effect  of  repeated  sequences. 
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Table  C6 

Array  (m  -  4)  Content  of  Repeated  Sequences  E:q>ressed 
In  Hexadecimaf  Notation 


Sequence  Number 
(from  Table  C4) 

Array  Content 
Hexadecimal 
Notation 
(see  Table  C7) 

Number  of  Digits 
per  period  (n,) 

1 

0 

1 

2 

f 

3.4 

5, a 

2 

5.6.7 

2,4.9 

3 

8.9.10 

6.b,d 

11-14 

1.2.4, 8 

4 

15-18 

3,6.9, c 

19-22 

7.b,d,e 

23-27 

0.1,2, 4.8 

S 

28-32 

1,3,6, 8, c 

33-37 

2.4, 9.9, a 

38-42 

3,7,9,c,e 

43-47 

9.6,a,b,d 

48-92 

7,b.d.e,f 

93-98 

0.1.2, 4, 8 

6 

99-61,69-67 

0.1,3,6,8,c 

92-64,98-70 

1.2.4.9.8.a 

71-76 

l)3p7|8|Cf  6 

77-79,83-89 

2,9,6, 9.b,c 

80-82,89-88 

2,3,6, 9,a,d 

89-94 

3,7,9,c,e,f 

99-100 

9,7,a,b,d,e 

101-109 

7,b,d,e,l 

107-113 

0,1,2,4,8 

7 

114-110 

0,l,3,6,8,c 

121-127 

0,l,2,4,9,8,a 

128-134 

1.2,4, 8,9 

139-141 

0,l,3,7,8,c,e 

142-148 

l,2,9,9,8,b,c 

149-199 

1, 3,4,9, a, d 

196-163 

2,3,4, 6, 9,c 

164-170 

2,4,5,9.a 

171-177 

l,3,7,8,c,e,f 

178-184 

*,5,7,9,b,c,e 

Table  C7 

Hexadecimal  Number  Equivalents 


Notation 

Binary 

Hexadecimal 

0000 

0 

0001 

1 

0010 

2 

0011 

3 

0100 

4 

0101 

5 

0110 

6 

0111 

7 

1000 

8 

1001 

9 

1010 

a 

1011 

b 

1100 

c 

1101 

d 

1110 

e 

1111 

f 

Table  C8 

Digital  Oroups  Associated  with  Some 
Specific  Orouping  Fbnctlons 


Binary 

Hexadecimal 

Measure  Group 
(equation  for 

0000 

0 

0001 

1 

0010 

2 

0100 

4 

1000 

a 

0011 

3 

0101 

9 

0110 

6 

8o*q> 

1001 

9 

1010 

a 

1100 

c 

0111 

7 

toil 

b 

4p^ 

1101 

d 

*  eo*q 

1110 

e 

1111 

f 
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